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ABSTRACT 

This report focuses on the School Action 
Effectiveness Study (SAES) , the national evaluation of the Office of 
Juvenile Justice and Delinquency Prevention's Alternative Education 
Program. It presents the history, start-up activities, successes, and 
problems encountered during the first year of the 17 action projects. 
In the first section, the report discusses general topics that 
summarize the entire evaluation. Chapter 2 discusses the record of 
research in creating organizational change and delinquency ^ 
prevention. Chapter 3 summarizes the conditions that lead to rigorous 
summative evaluation, while Chapter 4 describes the prograia 
development evaluation model used in the study. Chapter 5 presents 
some of the measures used in SAES to provide diagnostic information 
and to measure delinquency. Chapter 6 provides a brief sketch of the 
research designs for the 17 projects along with an overview of the 
formative evaluation results. The second section provides a narrative 
description of each project. A subject index, a name index, and a 
section containing abstracts of the narrative project descriptions 
are appended. (JAC) 
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Introductory Statement 



The Center for Social Organization of Schools has two primary objectives, 
to deveJop a scientific knowledge of how schools affect their students 
and to use this knowledge to develop better school practices and organi- 
zatlon. ^ 

The Center works through five programs to achieve its objectives The 
Studies i n School Desegregation program applies the basic theories ot 
social organization of schools to study the internal conditions of 
desegregated schools, the feasibility of alternative desegregation policies, 
and ?he interrelations of school desegregation with other equity issues 
such as housing and job desegregation. The School Organization Program 
is currently concerned with authority-control structures, task structures, 
reward systems, and peer group processes in schools. It has PJ°duced a 
large-scale study of the effects of open schools, has developed Student 
Team Learning Instructional processes for teaching various subjects in 
elementary and secondary schools, and has produced a computerized system 
for school-wide attendance monitoring. The School Process and Career 
Development program is studying transitions from high school to post 
se-onda^y in stitutions and the role of schooling in the development o. 
career plans and the actualization of labor market outcomes. The 
Studies in Delinquency and School Environments program is ^^^^^J^f ^. . ^, 
the Interaction of school environments, school experiences, and individual 
characteristics in relation to in-schopl and later-life delinquency. 

The Center ^Iso supports Fellowships In Education Research program that 
provides opportunities for talented young researchers to conduct and publish 
significant research, and to encourage the participation of women and 
minorities in research on education, 

This report, prepared by the Studies in Delinquency and School Environ- 
ments program, describes the interim results of the program ^s national 
evaJuation o? the Office of Juvenile Justice and Delinquency Prevention's 
(QJJDP's) Alternative Education Program, 
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The School Action Effec- 
tiveness Study (SAES) Is the 
national evaluation of the 
Office for Juvenile Justice 
and Delinquency Prevention's 
(OJJOP's) Alternative Educa- 
tion-Program. The study is 
rooted in the perception 
that neither social action 
nor research will make pro- 
gress without the collabora- 
tive effort of project 
implementers and researcti- 
ers. Together? these two 
groups can create change and 
examine its consequences in 
settings where answers are 
needed and probi ems are 
real • 

The study is also rooted 
in the notion that theory is 
an essential ingredient of 
both program development and 
evaluation research. Conse- 
quently t SAES aims to imple- 
ment an action research 
model f in which project 
implementers work together 
with researchers? specifying 
theory-based research ques- 
tions and designing their 
own evaluations as an aid to 
organizational self -study 
and continued project devel- 
opment* 

Any large evaluation pro- 
ject creates tension* Mul- 
tiple stakeholders? a col- 
lection of agencies and 
actors? and varied audiences 
with only partially overlap- 
ping--and sometimes diver- 
gent~interests present what 
will' always be a challenge 
for evaluators* Because 
evaluation necessarily 
involves critical and skep- 
tical examination? there nay 
be no way to mesh these 



divergent interests com- 
pletely* The approach taken 
in this pro ject is to 
involve as many parties as 
possible in the evaluation 
enterpri.se* When the sub- 
jects of evaluation are col- 
laborators in its develop- 
ment? the tensions may be 
reduced somewhat and evalua- 
tion feedback may be more 
helpful in project develop- 
ment* 

We have been partially 
successful wi th this 
approach? but complete suc- 
cess has escaped our graspo 
The reasons are many? and 
include limits on time? 
money? and our own talent 
and energy* Some of the 
more important reasons? how- 
ever? are differences in the 
perspectives of the Federal 
sponsor? the various action 
projects? and ourselves* 
The Venn diagram shown below 
illustrates the problem* 
The various actors in this 
project have sometimes over- 
lapping? and sometimes u- 
nique? goals or outlooks* 
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The primary sponsor 
(OJJDP) is first and fore- 
most interested in delin- 
quency prevention* It wants 
to evaluate demonstration 
projects--lnvol ving youth 
participation? organic a-* 
tional innovations? alterna- 
tive reward structures? and 
individual i /ed instruction 
(among other interven- 
tions )--cr eat i ng inst itu- 
tional changes that may pre- 
vent delinquency* The 
action projects sometimes 
place less emphasis on de- 
linquency prevention* They 
may be interested in educa- 
tional achievement or 
truancy? or in continuing a 
program already in exis- 
tence* Delinquency preven- 
tion is sometimes related to 
these interests even when it 
is not a primary aim* As 
evaludtors? we are inter- 
ested in assisting in the 
development of effective 
projects? critically assess- 
ing project effectiveness? 
and contributing to know- 
ledge* Our critiCvBl per- 
spective often results in 
approaches that diverge from 
the methods action projects 
find most comfortable* 
These projects often assume 
their interventions to be 
effective and prefer not to 
devote energy to the criti- 
cal and sometimes painful 
scrutiny of those activi- 
ties — especially in areas 
that are of interest to the 
sponsor and to sc ience but 
that may be seen as tangen- 
tial or even irrelevant by 
action project managers* 

Our bias when encounter- 
ing this tension has been to 
push as much as possible for 
a sound and thorough evalua- 
tion? in ways appropriate 



for each project* At the 
root of this bias Is the 
conviction that the public 
deserves educat ional and 
delinquency prevention 
efforts whose effectiveness 
has been or can be demon- 
strated* This is especially 
true in a Federal demonstra- 
tion program? in which the 
expenditure of public funds 
is justified by the evalua- 
tion of the resulting effort 
to learn how to develop and 
Implement similar projects 
effectively* In short? we 
have little sympathy for the 
arguments that evaluation 
diverts effort from or 
detracts from programmatic 
work in these circumstances* 
Because the effectiveness of 
these action projects is un- 
known? the only ethical 
course is systematic evalua- 
tion* In add it ion? we 
assume that the evaluation 
of a project's efforts? the 
generation of knowledge 
about the consequences of a 
project's inter vent i ons ? i s 
a part of sound project 
administration and implemen- 
tation* 

We have not assumed that 
evaluating this program will 
be easy? and we are grati- 
fied that we have been as 
successful as we have been 
in translating our ambitions 
into reality* The excellent 
rapport and cooperation we 
have with the Federal agen- 
cies involved? and with most 
of the action projects? have 
been critical in this suc- 
cess* 

This interim report sum- 
marizes some of what we have 
learned in the first year of 
the SAES* (For the most 
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part the report covers the 
period August 1980-August 
198l«) Most action projects 

M began implementing their 

interventions sometime dur- 
ing the past year? and all 

f 17 projects are now out of 

their planning phases* 
Start-up activities are now 
behind u-s: Action projects 
have had up to a year to 
become a^ustomed to our 
expectations and methods? 
and we have had up to a year 
to acquaint ourselves with 
the action projects. We are 
pleased that evaluation is 
becoming routinized as an 
expected and wel 1 -understood 
part of the activities of 
most projects. 

We are entering a second 
year of interaction with 
these 17 projects. In many 
casesf evaluation designs 
that are stronger than those 
possible in the first year 
are now being implemented. 
The next phase of this eval- 
uation should be more pro- 
ductive in assessing project 
effects on students and 
school s. 

Everyone is impatient for 
* information about a proj- 
ect's ••impact." This report 
is not the place to look for 
impact assessments. Here 
you will find information 
about a project's historyt 
its start-up activities? and 
its successes and problems 
in implementation during its 

* first year. You will find 
some organizational diag- 

* i noses f and some ideas about 
^ improving projects. Occa- 

sionallyt you will find pre- 
liminary attempts to assess 
effectiveness. These pre- 
liminary attempts are not— 
-nor are they intended to 



be--authoritative and con- 
clusive statements. They 
are intended to provide 
information useful for pro- 
ject devel opment. 

Effective projects 
develop over time? incorpo- 
rating feedback from their 
own observations and those 
of evaluators to become 
stronger. Provided that a 
project operates over a suf- 
ficient oeriod of timet with 
a stable set of goal s and 
guiding values? and with 
evidential pressure to guide 
the choice among alternative 
activities? an evolution 
that approaches an eventu- 
ally stable ••cliiiax" program 
may be expected (Tharp & 
Gallimoret n.d.). This 
report is therefore directed 
prinartly at project imple- 
menters and otherst includ- 
ing OJJDP and its technical 
assistance contractor? who 
have a stake in fostering 
project development or in 
planning new initiatives. 
It is also directed? how- 
ever? to the community of 
scientists engaged in the 
ei/aluatton of social pro- 
grams. The evaluation meth- 
ods being developed in the 
School Action Effectiveness 
Study should be of interest 
to evaluators and students 
of eval uati on. 

This interim report is 
tne product of many people's 
contributions? and it is 
possible to describe only 
some of these contributions 
here. Initial discussions 
leading to the proposal 
involved a small group: J. 
Douglas Grant suggested 
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using a program development 
approach and incorporating 
critical benchmarks as a 
component of the model* Oen- 
ise Gottfredson conceptual- 
ized the information systemt 
Lenore Campbell prepared a 
preliminary workplan on very 
short notice* and Gary Gott- 
fredson provided the theor- 
etical and methodological 
rationale for the project. 
This rationale was heavily 
influenced by the work of or 
discussions with LaMar 
Empey» Lee Sechrestt Joseph 
Meist David Hawklnst and 
Travis Hirschit among oth- 
ers. The general strategy 
owes much to the intellec- 
tual father of action 
researcht Kurt Lewint and is 
influenced by the example of 
Don M. Gottfredson in his 
work with criminal justice 
agencies. The evolution of 
the PDE model owes much to 
discussions with Carol Yama- 
saki* and the reactions of 
Social Action Research Cen- 
ter personnel in early staff 
training sessionst as well 
as to the the persuasive 
account of the ecology of 
program development and 
research provided by Roland 
Tharp and Ronald Gal li more. 

This project is based in 
part on the contributions of 
thousands of students and 
teachers who shared their 
views about their schools 
and provided information 
about themselves. We hope 
they will be rewarded for 
their help by having their 
views heeded and acted upon. 

Richard Carltont Deborah 
Oanielst Oenise C. Gottfred- 
sont Jane St. John* Deborah 
K. Ogawat Donald E. Rickertt 



Jr.f and Carol Yamasaki 
worked long and hard with 
action project personnel in 
workshops, site visits, and 
on the phone to prepare for 
tie surveys, to evolve Pro- 
gram Development Evaluation 
plans, and to draft project 
narratives. This report is 
possible because of their 
help and practical wisdom. 

Ann Sirdseye, Doris Coa- 
xum, Barbara Dilligard, 
Hilda Gutierrez, Roland Pat- 
terson, Edward N. Whitney, 
Glen Bader, Hilda Irwin, 
Richard Smith, Charles Almo, 
Herman S,teptoe, Thomas Cor- 
coran, Mary Lewis, Preston 
Elrod, Vanita Vactor, Anita 
Batisti, David Bailey, Ana- 
dia Andrews, Tyrone Seals, 
Phyllis Betz, Ciorah Montes, 
Ivelissi M. Chardon-Zavala, 
Craig Overbeck, Phiil ip 
Cano, Cristobal Lopez, 
Roberto Ouran, Nancy Cohen* 
Marilyn HcKnight, Prentice 
Oeadrick, Melvin Aaron, Dave 
Reiss, Lia Tepp, Joe Nathan, 
Mark Gilbert-Cougar, Nic 
Cooper, Sally Wisotzkey, 
Sonny Luster, William Kott- 
man, Tom Leighty, Oarlene 
Amyotte, Roy Mahoney, Pat 
Kenney, Marvin Dunn, Andrea 
Loring, Dolores Dunn, 
Michael Coleman, Pat Gahl , 
and Dorothy Martin were cri- 
tical links between the real 
world and the evaluation, 
whithout whom there would be 
no School Action Effective- 
ness Study. They provided 
the theories that guided 
much of the instrument and 
scale construction, devel- 
oped project plans using 
evaluation terminology, and 
made the action projects and 
data col 1 ecti on go. 
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Oenise C# Gottf rjedson 
performed the superhuman 

* task of coordinating and 

managing all of the datat 
Gary Gottfredson and Donald 

f E# RicK^rt designed the sur- 

vey instruments? and Debor ah 
K« Ogawa and Donald E« Rick* 
ert put i n many la-hour days 
analyzing data under incred- 
ible time pressure* Compu- 
tation at the Center for 
Social Organization of 
Schools is possible because 
of the rtork of Carol Wein- 
reich* 

Oelbert El 1 iott, LaMar 
Empeyt Joyce Ep^teint J« 
Douglas Grant? ~Joan Grants 
Travis Hirschit and John 
Holland gave valuable advice 
on the development of the 
student quest i onnai re? not 
all of it taken* Raul 
Romero translated the stu- 
dent questionnaire into 
Spanish? and Dennis Dillon 
of Intran Corporation pro- 
duced the optically scanna- 
bl e instruments* 

r 

Me were ably assisted in 
managing and processing the 
data by Stuart Gavurint 
Laura Gugertyt Richard 0# 
Joffet Helene Kapinost and 
Robert Kirschner* Helene 
Kapinos coordinated the flow 
of an incredible amount of 
information* Lois Hybl 
helped keep the project ord- 
erly despite what often 
seemed to be incessant new 

• demands f she maintained pro- 

ject files^ typed manu- 

^ .f scripts? and helped us to 

^ sthedule our time* 

Monserrate Diaz and Bar- 
bara Tatem of the Office for 
Juvenile Justice and Delin- 
quency Prevention cleared 



the way for thiE project to 
proceed? and helped to 
resolve nearly countless 
problems along the way* 

This report is edited by 
Gary D* Gottfredson? who was 
assisted by Claire Skarda* 
Ms* Skarda devoted many 
weeks to turning our nearly 
hope 1 ess or i g inal manu- 
scripts into grammatical and 
readable chapters* Gary D* 
Gottfredson wrote the chap- 
ters not ^attr ibuted to oth- 
ers* Michael Cook? Deborah 
Daniels? Denlse C. Gottfred- 
son? Lee Sechrest • and Jane 
St* John made useful com- 
ments on a draft of several 
chapters* Op ini ons 
expresssed are the authors' 
or editor's? and do not 
necessarily reflect the 
position or policy of any 
agency or institJtion* v.. 

The remainder of this 
report is organized into two 
sections* The first of 
tmese discusses general top- 
ics that undergird or summa- 
rize the entire evaluation* 
Chapter 2 discusses the 
record of research in creat- 
ing organizational change 
and delinquency prevention; 
i t summar i zes the ^^eak 
nature of foregoing effortsi 
and argues that the defects 
of these earlier efforts 
must be overcome to increase 
the value of research and 
demonstration programs* 
Chapter 3 summarizes the 
conditions that lead to rig- 
orous summative evaluation 
and argues that sone common 
objections to creating these 
CDnditions can be overcome* 
Chapter ^ describes the 
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approach taken by the School 
Action Effectiveness Studyt 
focusing on what we call the 
Program Development Evalua-* 
tion Model • Chapter 5 dis- 
cusses some of the measures 
used in SAES to provide the 
action projects with diag- 
nostic information? and to 
measure delinquency and the 
important theoretical inter- 
vening variables* Chapter 6 
provides a thumbnail sketch 
of the research designs for 
the 17 action projects? and 
summarizes the status of the 
evaluation and of the proj- 
ects themselves* An over- 
view of the results— all of 
which are formative rather 
than summative at the pres'^ 
ent stage of the evalua- 
tion—is also provided in 
Chapter &• 

The second section pro- 
vides a narrative descrip- 
tion of each action project* 
Most chapters were drafted 
by the field worker assigned 
to that project* Therefore? 
they generally have the 
benefit of having been given 
direct attention by the mem- 
ber of the evaluation team 
most familiar with the 
action project* At the same 
time? however? the involve- 



ment of multiple authors? 
each with a different back- 
ground and perspective on 
evaluation? has led to some 
unevenness in presentati on* 
S^me authors have striven to 
include as much information 
as they could to foster 
project development and to 
characterize the projects 
thoroughly* Others have 
leaned strongly toward pres- 
enting the projects with 
which they work in a posi- 
tive light? and have coped 
with the tension that could 
be created in the presenta- 
tion of constructive criti- 
cism by downplaying that 
aspect of the report* The 
editorial process cannot * 
eradicate the personal and 
stylistic differences that 
exist among the authors of 
these drafts* The reader is 
therefore urged to consider 
each of these narratives as 
a distinct essay? and to 
avoid making comparisons 
across projects on the basis 
of these individually 
drafted accounts* Many 
readers may be interested 
only in reading Part I- 
and then selectively dipping 
into chapters in the second 
section* 
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SAES Introduction 



Introduction to the School Action Effectiveness Study 



The Al ternat iyg^Educatj on 
Program 

The Office of Juvenile 
Justice and Oeltnquenty Pre- 
vention (OJJOP) has funded 
17 demon strati on projects as 
part of a Program in Delin- 
quency Prevention through 
Alternative Education* This 
OJJOP initiative is premised 
in part on the observation 
that individual delinquency 
is associated with a number 
of school -r el a ted or 
school -based probl emsf 
including disruptive class- 
room conduct? absentee! smt 
truancyt and dropout* An 
additional basis for foster- 
ing delinquency prevention 
through alternative educa- 
tion is found in a major 
theory of delinquency (Hir- 
schi f I969)f in which com- 
mitment to educational or 
other conventional goalst 
attachments to teachers and 
the school f and belief *in 
rules are viewed as bonds of 
social control which prevent 
delinquent behavior* Learn- 
ing theoryf especially 
social learning theory (Ban- 
dura? 1971Jf provides an 
explication of the ways in 
which these elements of the 
social bond may be strength- 
ened by appropriate educa- 
tional env i r onments* Soc i al 
learning theory also helps^^^ 
to explain how the influence 
of alternative school organ- 
izationf and the influence 
of peersf teachersf and 
parentSf can converge in 
preventing* or failing to 
preventf del inquency* These 
theor et i ca 1 per spec t i ves 
find substantial support in 



the evidence provided by 
research; they concur in 
i mpl y ing that al ter native 
education programs can be 
structured in ways that will 
reduce delinquent b^.havior 
(Gottfredsonf 19 81b) • 

This outcome--pr imary f 
and to a certain extent sec- 
ondary* prevention of delin- 
quency-might be achieved in 
al ter native educat i on pro- 
gr ams through thei r effects 
on the academic and social 
devel opment of the youth 
i nvol ved* 

The demons tr at i on progr am 
is for the most part tar- 
geted at school s servi ng 
grades 6 through 12 in rela- 
tively high crime communi- 
t i est wi th hi gh rates of 
del inquencyf dropoutf sus- 
pensions? expulsionsf absen- 
tee i smt and youth unempi oy- 
ment* Projects funded as 
part of this program were to 
be aimed at achieving 
(a) decreases in delinquent 
behavior in and around 
school Sf (b ) decreases i n 
dropouts* suspensions* 
expul sions* and truancy? 

(c) increases in attendance* 

(d) increases in academic 
success in school with con- 
sequent i ncr eases in gradua- 
tion rates* (e) improvements 
in the early post-school ing 
labor market experiences* or 
in the post-secondary train- 
ing or education* of youth 
assoc iated wi th participat- 

i ng school s * 

The achievement of these 
objectives re qui res some 
reorganization of school 
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pol iciest practices? and 
e nv i r oniient s • Spec i f i c al 1 y t 
the OJJOP program calls for 
achieving the folloi^iing 
instrumental objectives to 
foster the attainment of the 
overarching program goals: 
(a) limiting or decreasing 
referrals to the juvenile 
justice system; (b) making 
school discipline fair and 
consistent while providing 
for due process; 

(c) increasing youth? 
parent? and community agency 
participation in school 
decision making to reduce 
student alienation and feel- 
ings of powerl essness f 

(d) decreasing the grouping 
of students according to 
inappropriate criteria {such 
as social class or race) 
which? accompanied by 
improved learning environ- 
ments? should preclude 
labeling effects and stigma- 
tization while enhancing 
educational success; and 

(e) providing a structure 
for learning that promotes 
educational and social 
development because it is 
tailored to realistic levels 
of performance for indivi- 
dual students* 

A number of national 
advisory panels (President's 
Science Advisory Committee 
on Youth? 1973; Carnegie 
Council on Policy Studies in 
Higher Education? 1979; Pre- 
sident's Commission on Law 
Enforcement and Administra- 
tion of Justice? 1967) and 
del inquency researchers 
(Gold? 1978; Gottfredson? 
1981a; Hawkins & Weis? 1980; 
Johnson? Bird? £ Little? 
1979; Hawkins t Mall? 1979) 
have argued that tradition- 
ally organized schooling? 



which focuses primarily on 
academic goals? does not 
provide opportunities to 
develop cognitive? interper- 
sonal? and vocational compe- 
tencies and may be inappro- 
priate for many of today's 
students* Incorporating 
alternative educational 
options into school programs 
should provide more opportu- 
nity for development of such 
competencies or a better fit 
between student and school? 
tHus promoting post-school- 
ing vocational adjustment 
and preventing delinquency* 
Several authors (including 
Hawkins 6 Mall? 1979; Gott- 
fredson? 1981a; McPartland £ 
HcOill? 1977; Johnson et 
al«? 1979) have stressed the 
inability of conventional 
school reward structures to 
enable all students to 
experience success. This 
outcome — a failure of many 
students to be rewarded in 
school — decreases their 
stakes in conformity by 
decreasing their attachment 
to school and their commit- 
ment to educational goals. 
Youths who do not find 
school rewarding have little 
reason to conform. Conse- 
quently? alternative reward 
structures are one important, 
feature of the alternative 
education provided in the 
action programs. 

The OJJOP (1980) program 
announcement invited appli- 
cations for action projects 
intended to alter school 
organization? climate? and 
educational practices. Spe- 
cifically? the following 
characteristics were 
requested in the solicita- 
tion: 
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!• Projects are to pro- 
vide opportunities for 
voluntary participation in 
al ternati ve educati onal 
experiences aimed at ••devel- 
oping constructive interests 
relevant to (youths envi- 
ronment'* and promoting edu- 
c at i onal and psychosoc i al 
development* The focus of 
projects is to be on youth 
making transitions from ele- 
mentary school to junior 
high and from junior to sen- 
ior high school? although 
this focus need not be 
excl usi ve« Tracking? 1 abel - 
ing? segregation? and stig- 
mati zat ion of students i s to 
be avoided* 

2* Projects are to spe- 
cify goals and objectives 
aimed at achieving the OJJOP 
program goals and objectives 
discussed earlier* 

3* Projects are to exem- 
plify the following elements 
thought to be conducive to 
achieving these goals: 
< a) individual i zed instruc- 
tion; (b) clear reward 
structures that supercede or 
supplement traditional 
classroom grades and that 
reward student improvement? 
incorporating a flexible 
array of rewards for differ- 
ent amounts of pr ogres s; 

(c) coalitions of school? 
communi ty? bus i ness? parent? 
and youth leadership that 
attempt to change the educa- 
tional envi ronment ; 

( d) comprehensive rather 
than piecemeal attempts to 
improve schools? involving a 
variety of rel evant organi- 
zations and agenc ies; 

<e) use of peer and parental 
influence as a vehicle for 
accomplishing goals; (f) the 



training of school personnel 
and the creation and imple- 
mentation of practices to 
increase posi ti ve i nterac- 
t i on with and responsiveness 
to students; (g) small pro- 
gram size and favorable stu- 
dent- to- adul t rati OS ; 
( h) strong? fair? con sis tent 
school governance and admi n- 
istration devoted to student 
growth; and (i) caring? 
competent teachers* 

These OJ JDP-gener ated 
project speci f ications 
constitute the first of 
three bases for an evalua- 
tion* The second basis is 
the theories of action which 
under 1 y the project- speci f i c 
goals? objectives? and 
i ntervention model s each 
project devel ops* The thi r d 
basis for the evaluation is 
the broader delinquency pre- 
vention and educational lit- 
erature? which specifies 
some intermediary objectives 
that are important for 
del inquency prevent i on 
efforts* 

IX^lliJItion^Aiins 

The overarching goal of 
the-^chool Act i on Ef f ecti ve- 
ness Study is to create coij- 
mynic atabl^e knowledge about 
del inquency prevent i on 
theory and practice* But a 
complex evaluation such as 
the School Action Effective- 
ness Study must accomplish 
many aims if it is to be 
effective* There are many 
audiences i nterested i n 
delinquency prevention? but 
so far demonstration and 
evaluation efforts in the 
delinquency prevention area 
have amassed a dismal 
record* 



SA6S Introduction 



The audience for the 
study includes project man- 
agers and their organiza- 
tionsv because they want (or 
sometimes need but do not 
want) feedback on their pro- 
gress as one tool to use in 
developing their projects. 
The Federal sponsor is 
another audience with a 
direct and immediate inter- 
est in the evaluation 
because it has chosen alter- 
native education as a prom- 
ising area for research and 
development* The sponsor 
mission is to contribute to 
knowledge in delinquency 
prevention and to develop 
prevention methods that can 
be suggested for broader 
implementation; it needs 
evaluation to accomplish 
this mission* The general 
publ iCf keenly aware of what 
it perceives as widespread 
youth crime and disorderly 
school Sf is a third audi- 
encet with both prurient and 
practical interest in a 
problem that affects every- 
day .1 ife* Evaluation 
researchers are struggling 
to develop paradigms for 
evaluation under difficult 
circumstances? and they are 
therefore eager to learn 
what others at the forefront 
of evaluation are doing* 
Researchers and theoreti- 
cians in sociology? psychol- 
ogy? and criminology have a 
direct and obvious interest 
in the knowledge gained 
through action research. 
Finally? Congress and? pos- 
sibly? state and local leg- 
islative bodies want to know 
how effectively the public^s 
funds are being spent? and 
what kinds of programs they 
should support in the 
future? Legislators and 



other policy makers are 
therefore important audi- 
ences* 

As the chapter by Ogawa 
(this volume) nakes clear? 
previous delinquency preven- 
tion efforts and their eval- 
uations have been fraught 
with problems of incomplete 
implementation? weak evalua- 
tions? and lack of interme- 
diary and outcome measures 
required to assess the 
efforts* Not only delin- 
quency prevention programs 
suffer from these problems* 
Sarason (1971) describes the 
disappointing degree of 
implementation of attempted 
educational innovations such 
as the ••new math*** Whereas 
the developers of the inno- 
vation intended to alter the 
ways teachers interact with 
students? the ma j ar outcome 
was the use of some new math 
books* Lots of educational 
evaluations are? as Charters 
and Jones (1*^73) put it? 
evaluations of ••non-events**' 
The SAES must take steps to 
avoid evaluating non-events? 
and also to avoid the other 
problems from which earlier 
prevention evaluations have 
suffered* 

The multiple consumers of 
this evaluation and the his- 
tory of previous delinquency 
prevention efforts imply 
that SAES should accomplish 
the following goals: 

!• The collection of 
sound measures of delinquent 
behavi or? achievement? 
attendance? persistence in 
school? and vocational 
behavior? as well as meas- 
ures of the relevant inter- 
vening theoretical variables 
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believed to be associated 
with these outcomes* 

2« The establishment of 
evaluation designs that 
allow the wost confident 
interpretation of results 
poss tb1e« 

3« The documentation of 
project history? context? 
and conduct* 

^« Documentation of the 
theoretical rationale under- 
lying each project's inter- 
ventionsf and assessment of 
the plausibility of that 
rati onal e« 

5m Documentation of 
project implementation in a 
way that allows assessment 
of the strength and integr- 
ity of that implementationt 
and replication of the 
interventions if warranted* 

bm The development of 
sound project management 
plans to increase the effec- 
tiveness of each project and 
to help accomplish Goal 5« 

?• The education of 
project managers in the 
utility of evaluation as a 
management tool and in the 
use of feedback about proj- 
ect implementation and 
effectiveness to foster 
project development* 

d« The development of 
knowledge about conducting 
an evaluation* 

9m The development of 
fundamental knowledge about 
del inquency prevention and j 
educ ati on« 



10# Explicit guidance 
for pol icy makers and future 
program i mpl ementer s to 
increase the payoff from 
future expenditures of 
publ ic funds* 

These are the overarching 
goal s of the School Act i on 
Effectiveness Study* These 
goals are being pursued in 
the context of a Federal 
research and devel op men t 
effort which provides 
further focus and structure 
for SAES* The remainder of 
this chaoter explains some 
of the specific concerns of 
the evaluation and describes 
the interorganizati onal con- 
text of the evaljation* 

§^§S_C oncer ns 

Pro ject^Exgec tat i, ons_about 
Eyal uati on 

o The OJJDP program 
announcement spec i f i ed that 
action projects must explain 
how the i r app roach woul d 
enable a national evalua- 
tion* It also required 
applicants to give assu- 
rances of their willir^gness 
to cooperate with a manage- 
ment information system* 
F urthermoret an append i x to 
the program announcement 
discussed the desirable fea- 
tures of an e va I uat i on« The 
action projects selectedt 
howevert did not usually 
address these issues thor- 
oughly* Some projects were 
surprised that they would be 
i nvol ved in ser ious summa- 
tive evaluation^ and would 
be asked to c reate arrange- 
ments to increase their 
projects* evaluatabil ity* 
Other projects did not anti- 
cipate (despite the program 
announcement) that they 
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Mould be collaborating with 
a national evaluator. Con* 
siderable work uas therefore 
required to (a) explain the 
elenents of formative and 
suwmative evaluation to 
action project staffs? 
(b) obtain staff cooperation 
and gain access to the 
information required to con- 
duct these evaluation activ- 
ities? (c) demonstrate that 
evaluation can be helpful to 
project impl ementersf and 
(d) negotiate arrangements 
to increase project evaluate 
ability* Unf or tunatel y» the 
legacy of many previous so- 
called evaluations in the 
education and delinquency 
prevention fields is one of 
extensive miseducation about 
evaluation issues* Conse- 
quently? important short- 
term objectives for the SAES 
involved orientating action 
project staffs to a serious 
evaluation aimed not only at 
making hard-headed summative 
assessments of their proj- 
ects? but also at assisting 
in project development* 

The history of evaluation 
research in delinquency pre- 
vention is replete with 
examples of programs in 
which the implementation was 
undocumented or not carried 
out as planned (Oixon t 
Wright? 1974; Krisberg? 
1978; Ogawa? this volume)* 
Knowing the fidelity with 
which program plans are 
implemented? the strength of 
the educational and social 
"treatment?" and the context 
within which the program 
operates is essential for 
three reasons* First? any 
evaluation result— either 
positive or negative— is of 



little value unless the 
nature of the program is 
wel 1 described* Second? 
information derived from 
monitoring the activities 
and the implementation of 
plans is needed to 
strengthen the integrity of 
trie program? and to detect 
unforseen consequences or 
potential breakdos^ns in 
project plans or the evalua- 
tion des i gn* Thi r d? nega- 
ti'^ e results of summative 
evaluations have sometimes 
led observers to conclude 
that the interventions 
intended to be implemented 
do not work? whereas the 
interventions may not in 
fact have been implemented? 
implying a quite different 
conclusion* Knowledge of 
what was actually imple- 
mented is essential in draw- 
ing conclusions from tests 
of any planned intervention* 

Project ^environment* A 
C3mponent"of 'our work has 
been to describe the origins 
and development of the 
action projects* This 
includes a history of the 
practical and theoretical 
origins of the projects? 
accounts of the po3ulations 
served? and description of 
the links between the 
schools or school systems 
and other agencies* 

Strength_and_i.nte3ritjt_of 
B1 anned^interjfentions* 
Assessment^'of the planning 
and implementation process 
consists of two distinct 
components (Sechrest? West? 
Phillips? Redner? t Yeaton? 
1979)* The first relates to 
considerations of the 
strength of the intervention 
pTan* This is essentially a 
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matter of the construct 
validity of the measures 
intende d to be^taken in an 
intervention* In a medical 
analogyt if a person is suf- 
fering from a bacterial 
infection? treatment with a 
sufficient dose of an anti- 
bacterial agent may be 
deemed a construe t-val id 
(and strong) treatment* 
Treating the same person 
with aspirin (in whatever 
dosage) would be deemed a 
weak treatment 1 acking in 
construct validity* No 
rules have been agreed upon 
for assessing the strength 
of programs such as the 
al ternat i ve educat ion acti on 
projects* Several proce- 
dures are available? how- 
ever* These include 
(a) analysis of the plausi- 
bility of the plans* theo- 
retical premises? ond deter- 
mination of how closely the 
specifics of the plans are 
linked to delinquency pre- 
vention theories; (b) expert 
Judgments about the likeli- 
hood that the project as 
specified will produce the 
desired outcomes; and 
(c) comparisons of the 
intended programs with the 
range of current or past 
efforts at delinquency pre- 
vention (in this way a pro- 
gram that was otherwise 
unremarkable but resembled a 
previous inevfective effort 
mi ght be judged a weak pro- 
gram)* In addition to a 
theoretical basi s? parame- 
ters involved in making 
assessments of strength 
include staff stability or 
qual if ications? intensity 
and durat i on of treatment ? 
focus of effort? clarity of 
plans? and the extent to 
which the olans involve dif- 



ferent responses to 
di f f erent per sons ( e* g* ? 
indi vidua 1 i zed instruction) * 
In general? replications of 
previously tested or well 
engineered intervent i ons? 
comprehensive attempts to 
cope witi the mul t i pi e 
causes of a probl em? treat- 
ments with clearly spelled 
out treatment protocols or 
i mpl ementdt i on manual s? or 
pr i mary prevent ion ef forts 
that affect a substantial 
proportion of an environ- 
ment's inhabitants are 
likely to be judged stronger 
than those that lack these 
characteristics* 

The second aspect of 
assessing program implemen- 
tation relates to the iQteg- 
tLll fidelity with which 
pi ans are i mplemented* 
Clear plans are more likely 
to be implemented with 
fidelity than diffuse plans? 
fuzzy promi ses? or vague 
project descriptions* Some 
components of imp I ementat ion 
that must be monitored or 
observed are (a) staf f i ng 
patterns (including experi- 
ence? training? numbers? and 
stabi 1 i ty ) ? ( b) net hods used 
to select? admit? or reject 
the youth involved in each 
project and each of its com- 
ponents? (c ) the di f f eren- 
tial assignment of youth to 
alternative p rogr ams? or the 
basis for individualization 
of instruction? (d) the 
nature? durat ion? c i rcums- 
tances? and frequency of 
services to individuals or 
groups? ( e) methods used to 
determine who ( incl uding 
students) is involved in 
i mpl ementat i on? (f ) the 
i ntervent ions* elements and 
their duration? (g) the 
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degree of project staff 
comitnentf (h) project 
supervisory and management 
practices? and 
(i) curricular material St 
indi viduaH zed education 
plansf lesson plans? diag* 
nostic protocols? treatment 
plans? and the like* 

The importance of this 
aspect of assessing imple- 
mentation can scarcely be 
overestimated* The scope of 
the alternative education 
action projects? encompass- 
ing as they do many distinct 
components? makes the faith- 
ful implementation of all 
plans unlikely* * failure 
to obtain sound evidence 
about the strength and 
integrity of these preven- 
tion projects could lead to 
erroneous concl us ions about 
the efficacy of the del in-^ 
quency prevention ideas 
behind these projects* It 
could.be a mistake? for 
example? to conclude that 
reorganized educational 
reward structures do not 
help in preventing delin- 
quency (or in promoting 
career development) on the 
basis of negative summative 
evaluation results* Speci- 
fically? this conclusion 
could be a mistake if there 
were no solid evidence that 
reward structures were actu- 
ally altered in systematic 
ways* Equal ly important? 
even if a summative evalua- 
tion implied that a project 
had been effective? in the 
absence of sound information 
about what actually was done 
the project would provide 
little basis for its repli- 
cation at a new site* Such 
a project would provide only 
the shakiest guide to others 
who wish to implement a 



similar program* 

t icesy ,Proced ur eS£_and_their 

IfiBliciiiiai" 

The OJJOP Alternative 
Education Program hopes to 
alter school policies and 
practices dealing i#ith droo- 
outs? school di srupt ion? and 
del inquency? and to deter- 
mine the implications of 
those changes for the school 
and its students* Empiri* 
cal ? theoretical? and prac-* 
t ical cons i derations (Gotf 
fredson C Daiger? 1979; 
National Institute of Educa- 
tion? 1978; Toby? 1980; 
Howard? 1978) implicate poor 
or inconsistent school dis- 
ciplinary practices in the 
failure to prevent disrup- 
tion in schools* Further- 
more? evidence implies that 
youths who will drop out of 
school are more often disci- 
plinary problems and experi- 
ence more absolute or rela- 
tive academic failure while 
still in school (Elliott L 
Voss? 197^; Hirschi? 1969; 
Got tf reds on? 1981 a; Hawkins 
£ Weis? 1980; Johnson? 1977; 
Gold* 1978)* School prac- 
tices? policies? and proce- 
dures for coping with or 
responding to disruptive 
behavior — especially the 
fairness? firmness? and con- 
sistency of with ftfhich rules 
are applied--are of great 
importance in preventing 
delinquent behavior and 
other forms of misconduct* 
Improvements in this area 
may be expected to pay off 
in terms of reduced delin- 
quency* Si mi larl y? altering 
school s* responses (McPart- 
land C HcOill? 1977) to 
youth who have difficulty in 
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coping with traditional aca- 
demic programs (by providing 
individualized curricula? by 
rei#arding and encouraging 
the development of a wider 
variety of social Skills and 
vocationally related compe« 
tenciesf and providing a 
more extensive range of 
rewards and responses) may 
be expected to increase 
learning? promote psychoso^-* 
cial development? and 
decrease delinquency and 
dropout* Thusf improved . 
school governance? altered 
curricula? and increased 
responsibility may all 
directly or indirectly lead 
to decreased student miscon«- 
duct and increased school 
retention rates* These 
changes? if they occur? must 
be documented by the SAES* 

Youth and Par ent Partici pa- 
tion 

The evaluation aims to 
determine the effect of 
action programs on youth and 
parent participation in 
school activities* An 
increase in such participa- 
tion is expected to prevent 
delinquent behavior* 

In Hirschi*s (1969) theo- 
retical account? youth 
involvement in conventional 
activities and commitment to 
convent i onal goals or pur- 
suits are important bonds to 
society which serve to con- 
trol behavior* And? youth 
involvement in school activ- 
ities carries with it the 
opportunity for increased 
interaction with peers and 
teachers? an outcome that 
also may serve to increase 
stakes in conformity* Hir- 
schi (1969) marshal Is some 



empirical support of this 
theoretical perspective* 

Parental involvement in 
school activities may also 
have salutory effects* 
Recent reviews of the use of 
home-based reinforce^s as an 
aid to the classroom manage- 
ment of disruptive behavior 
(Barth? 1979; Atkeson G 
Forehand? 1979) imply that 
cooperation of parents in 
providing backup reinforcers 
is useful* Gaining that 
cooperation is a major prac- 
tical problem* in additiont 
ample testimonial evidence 
(NcPartland e McOillf 1977; 
Hawkins t Hallf 1979? p* 25) 
implies that parent involve- 
ment may be important* 

The efficacy of increased 
parental or student involve- 
ment in school decision. mak- 
ing is more dubious* Gott- 
fredson and Oaiger (1979) f 
in a reanalysis of the Safe 
School Study data? conclude 
that no evidence that such 
participation is related to 
school disruption exists in 
that study of over 600 
schools* They accord with 
the original (National Insti- 
tute of Education (1978| and 
Hawkins and Wall (1979) 
assessments in this regard* 
Despite considerable testi-* 
monial evidence that such 
participation may be impor- 
tant (summarized in Hawkins 
e Wall? 1979)? little firm 
evidence or carefully arti- 
cjlated theory implies that 
student or parental partici- 
pation in decjsion^maHjng is 
a promising strategy to 
reduce delinquency* Availa- 
ble evidence is based? how- 
ever? on the analysis of 
natural var iati on* Because 
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schools typically do not 
involve students in major 
ways in decision makingt 

. this evidence does not show 

the consequences of major 
youth involvement in deci- 

t :-sion making* 

The notion that student 
participation in conven- 
tional activities such as 
athleticsf band? clubs? stu- 
dent government? and the 
like may prevent delinquency 
has more support in the lit* 
erature* Here? theory (Hir- 
schlf 1969) and research 
(Gottfredson t Oaigert 1979) 
converge in implying that 
such participation may be 
Important* Indeed? typical 
explanations of the often- 
observed association between 
school size and delinquency 
(Hawkins t Weist 1980) 
Involve arguments about the 
lowered opportunity for par- 
ticipation or Involvement In 
large schools (Garbarinot 
1978; McPartland L McDlUt 
1977; Wiatrowskif Gottfrcd- 
sonf C SwatkOf 1980)* 

Youth and parent partici- 
pation is an intermediate 
outcome* As sucht it could 
be considered a measure of 
the strength and fidelity of 
an intervention* The over- 
all goal of reduced delin- 
quency is expected to come 
about as a result of 
increased participation* 
Thusf there are two Impor- 
tant evaluation questions: 
(a) To what degree are proj- 
ects characterized by stu- 
j dent and parent partlclpa- 

tion? (b) Does 
participation appear to con- 
tribute to delinquency 
reductlonf and to decreases 
In truancy and absenteeism? 
An ancillary set of 



questions related to the 
second of these Is whether 
Increased participation pro- 
duces the theoretically 
expected increase In attach- 
ment? comml tr sntf and 
bel lef * 

School^Achigvementl Social 

AcadimXcilandlvocational 

Vgcatj^aT Be ha vLor 

One set of objectives for 
the evaluation is to deter- 
mine the effects of the 
alternative education pro- 
grams on (a) educational 
performance? (b) social t 
academic? and vocational 
devel opment ? and (c) the 
transition between secondary 
education and work? post- 
secondary education? or - 
vocational training* 

Experimental evidence and 
theory predict that altered 
reward structures will 
I nf luence educational out- 
comes* Sped f ical 1 y? Inter- 
ventions involving the reor- 
ganization of academic 
rewards? so that all stu- 
dents are rewarded in pro- 
portion to their educational 
Improvement rather than in 
accordance #<lth their per- 
formance relative to other 
students? hold much promise* 
Slavln (1980) recently 
reviewed the literature 
showing that cooperative 
team learning Is a powerful 
way of nariesslng peer group 
interaction to promote 
learning at the same time 
that it Improves students^ 
satisfaction wl th the educa- 
tional process and- Increases 
learning according to stan- 
dardized achievement tests* 
Various strategies are 
described in the 
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experimental literature Sla- 
vin discusses? but. they have 
two themes in common* 
Firstf groups of students-- 
teams or c1 assrooms— are 
rewarded in some way on the 
basis of group performance* 
Secondf each student can 
contribute to the perfor- 
mance of the group regard- 
less of his or her current 
level of academic perfor- 
mance* This is accomplished 
either by assigning points 
to the team based on 
improvement s in individual 
performance above each stu- 
dent's baSwIine performance? 
or by structuring competi- 
tion so that students of 
approximately equal ability 
compete with each other* 
Points are then credited to 
the te am based on this 
str»jctured competition* 
Because students compete 
with others of approximately 
equal ability* all students 
contribute points to the 
team in approximately equal 
proportion* This is in 
sharp contrast to the tradi- 
tional classroom system in 
which some students never 
are rewarded or perce i ve . 
themselves as contributing 
to the performance of a 
valued peer group or class* 

Such learning structures 
have never Deen evaluated 
for their effects on delin- 
quent or disruptive behav- 
ior* But theory predicts 
that such programSf which 
resemble what the QJJOP has 
called for in its Alterna- 
tive Education Program* will 
reduce delinquency if imple- 
mented with sufficient 
strength and fidelity* The 
existing evaluations of 
these programs show 



(a) increased academic 
performance* (b) increased 
sel f -esteem* (c) increased 
peer friend sh i ps* and 
(d) increased satisfaction 
with school* When trans- 
lated into Gold's (1978) and 
Mir schi • s (1969) theoretical 
terms* these outcomes i mpl y 
increased self-esteem (lead- 
ing to decreased need to use 
del inquent behavior as an 
ego-defense) * and i ncr eased 
attachment to schools and to 
peers* Johnson et al * 
(1979) summarize additional 
evidence that the kinds of 
peer group and reward struc- 
tures created by such inter- 
ventions may be effective* 

Some action projects are 
attempting to "indi vidual- 
ize** instruction by using 
di f f erent ial educational 
treatments* such as alterna- 
tive curricula or teaching 
styles* or by developing 
learning plans based on an 
individual di agnosis* A 
recent comprehensive review 
by Cronbach and Snow (1977)* 
which analyzes work seeking 
to establish and use know- 
1 edge about i nter act ions 
between student characteris- 
tics and i nstruc ti onal 
treatments* confirms the 
utility of this approach* 
although progress in this 
area is not as great as 
woul d be hoped* Attempts at 
**individual izing" instruc- 
tion* based on research by 
Fizzell (1979) or by Hunt 
(1974)* are described in the 
nJJOP solicitation's back- 
ground paper by Hawkins and 
Wall (1979); these attempts 
are examples of strategies 
that programs could attempt 
to implement and which show 
at 1 east some promi se* 



SAES I ntroduct i on 



The evaluation task is 
threefold:^ to describe and 
document the implementation 
of the interventions used by 
the action projects? to 
assess the contribution of 
1 these activities to student 

academic performance? and to 
assess the contribution of 
improved academic perfor- 
mance or skills to delin- 
quency redact i on« 

Vocational and Educational 
Dey/cgloement 

These and other aspects 
of the alternative education 
projects may influence 
social and vocational 
skills* Altered student 
participation rates in a 
variety of school activi- 
ties? more experience with a 
broader range of curriculum 
(some of which is directed 
to vocational and interper- 
sonal development)? and the 
altered nature of peer group 
interaction and reward 
structure may all contribute 
in some degree to these out- 
comes. Krumboltz (1978)? 
for example? has spelled out 
a social learning theory of 
vocational development which 
implies that such influences 
should alter individual com- 
petencies and inclinations 
to pursue various careers. 
The devel op men t of voca- 
t ional and interpersonal 
skills should increase 
youths' stakes in conformr 
ity? and thus prevent delin- 
quency. 

The task of assessing the 
effects of these projects on 
the transition from secon- 
dary school to post-secon- 
dary employmentt trainlnq? 
or education is a difficult 
one. Longitudinal studies 
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extending beyond the antici- 
pated three- or four-year 
duration of the evaluation 
would be helpful here. The 
reason for this is that only 
rel at ivel y few students will 
have experienced a project 
for three years and accumu- 
lated any post-secondary 
work? training? or educa- 
tional experience in this 
time span. For students 
ex per ienci ng fewer than 
three years of an alterna- 
tive education program? the 
intervention will probably 
lack sufficient strength to 
produce substantial effects. 
Despite these limitations? 
evaluation tasks i ncl ude 
documenti ng? insofar as is 
possible? the educational 
and vocational plans of stu- 
dents leaving secondary 
school? and assessing the 
contribjtion of various pro- 
gram components on those 
early career outcorr<es. 

Del irjauenc^ 

Assessment of the effects 
of the alternative education 
projects on rates of drop- 
out? suspensions? expul- 
sions? truancy? and del in- 
quency is a major goal of 
the evaluation. Improved 
educational exper i ences as a 
result of the alternative 
education programs? i f 
implemented with sufficient 
strength and integrity? 
should influence these out- 
C3mes. 

Academic performance is a 
strong correlate of delin- 
quent behavior in and out of 
school (Bachman? □•Mai 1 ey? & 
Johnston? 1978; Elliott t 
Voss? 1974; Empey G Lubeck? 
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1971; Hawkins t Weisf 1930; 
Gottfredsorif 1981a) • 
Truancy and dropping out of 
school also appear to form 
part of a constellation of 
behavior of which delin- 
quency is a frequent concom- 
mitant* Interventions that 
prevent delinquency may also 
be expected to influence 
these outcomeSfj^ both on the 
basis of empirical evidence 
and on the oasis of theory 
(Hirschif 1969)f which pos- 
tulates that attachment to 
school is an important 
ingredient in delinquency 
prevent ioh« 

Program Modgls^Moat^Ef f PC- 

Youth Under_OXf fgrent^CnnrtT* 
tj^ons " " * " ~ 

The evaluation also seeks 
to determine which types of 
alternative education models 
appear most effective for 
different types of youths? 
and under what conditions* 
This is a challenging task. 
Clear and confident answers 
to this set of evaluation 
questions will almost cer- 
tainly not be forthcoming* 
We are limited to informa- 
tion derived from 17 proj- 
ects? which are attempting 
to implement different 
i nterventionsf with differ- 
ing degrees of fidelity? and 
which are serving popul a* 
tions that differ* Most of 
these projects have not been 
set up to permit the unam- 
biguous search for the kind 
of statistical interactions 
demanded by these evaluation 
questions* And the history 
of the search for statisti- 
cal interactions in quasi- 
experimental research is a 
discouraging one (0* Gott- 
fredsont 1981)* 



Despite these difficul- 
ties? it is undoubtedly 
worthwhile to dredge the 
evidence from the 17 action 
projects for clues about 
what works best? for whoii7t 
under what conditions* Cer- 
tain interventions are most 
effective for certain tyoes 
of individuals* '=or exam- 
ple? a youth who perfornis 
poorly in school is expected 
to receive few rewards from 
traditional education and 
thus to have low attachment 
to school and little commit- 
ment to traditional educa- 
tional goals* Empirical 
evidence supports this gen- 
eralization (Hirschi? 1969; 
Sewell? Mailer? f. ^ortes? 
1969; 3achman et al*? 1978)* 
Other students? because they 
receive rewards and perform 
well in traditional school- 
ing? are already attached to 
school and committed to eau- 
cational goalso a program 
designed to alter reward 
structures may be effective 
in decreasing delinquency? 
truancy? and dropout among 
the former group put may 
Derhaps have no effect on 
the latter group* 

This is Dut one example 
of the theory-derived expec- 
tations that can guide a 
search for interaction 
effects. Other theories 
appear to predict that 
enhancing sel f-esteem 
through alternative educa- 
tion may be more important 
for youths with little 
social control than for 
those with high levels of 
SDcial control (cf* Gold? 
1978)« A thorough theoreti- 
cal approach (cf* Glaser? 
1977) to the searcn for 
interventions most effective 
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for particular subgroups 
appears to be the most 
fruitful way of pursuing 
this ev<Eiluation goal* 

The "under what condi- 
tions" part of this evalua- 
tion question hinges on 
issues of implementation* 
Projects with the most plau- 
sibility (or in the terms 
used earlier? the most 
strength? and those which 
are Implemented with most 
f idel ity) t will 1 ikely 
create the "condi tions" that 
are most effective* Other 
conditions that merit scru- 
tiny have been mentioned 
earlier* They include 
staffing patterns and sta- 
bility? resources? exten- 
siveness and duration of 
services? community charac- 
teristics? and the interor- 
gani zat i onal envi ronment 
within which the project 
operates* Learning about 
the conditions necessary to 
create effects is likely to 
involve a boot-strap opera- 
tion? in which clues to the 
conditions necessary come 
from the theory-based exami- 
nation of the nature of 
ef fecti ve interventions* 

E va 1 uat i o n ? , the ^&oi}lo£l,^a!l^ 
the Action Projects 

The Alternative Education 
Program is sponsored by the 
Office of Juvenile Justice 
and Delinquency Prevention? 
with supplemental funding 
provided through OJJOP by 
the Department of Labor* 
Three divisions of OJJDP are 
involved directly in this 
program* First? the Special 
Emphasis Division has pro- 
grammatic responsibility for 
the grant awards made to the 
17 action projects listed in 



Tables I and 2* Second? the 
Technical Assistance ^nd 
Training Division has 
responsibility for providing 
assistance in project devel- 
opment? and works through 
CDntractors to do so* Ini- 
tially? the Westinghouse 
National Issues Zenter was 
assigned these technical 
assistance tasks as part of 
its larger contract to pro- 
vide assistance for OJJDP's 
Del inquency Prevention 
Research and Development 
efforts* In recent months? 
Westinghouse was replaced by 
Polaris Research and Devel- 
opment in this role* Third? 
the National Institute for 
Jjvenile Justice and Delin- 
quency Prevention is respon- 
sible for the evaluation* 
Tne Institute nade a grant 
to the Johns Hopkins Univer- 
sity to perform this evalua- 
tion? and the University 
subcontracted part of the 
work to its col labor aOr? 
the Social Action Research 
Center* In shortt a total 
of 23 organizational enti- 
ties are di recti y i nvol ved 
in this effort. The parti- 
cipation of each i s essen- 
tial to the successful con- 
duct of the evaluation* 

The degree of collabora- 
tion and cooperation among 
these groups has Deen exem- 
plary* A major difficulty 
facing many evaluations is 
rivalry? or a lack of coor- 
dination among the various 
agencies involved* In this 
evaluati on? however ? the 
staffs of OJJDP^s Institute? 
Spec i al Emphas i s 0 i v i si on? 
and Technical Assistance and 
Training Division have met 
frequently with us and with 
Polaris to coordinate 
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activitiesf assist in each 
other^s efforts? share 
i nf orfitat ion? discuss prob- 
lemsf and plan solutions* 
This coordination has led to 
some blurring of the action 
projects* perceptions of the 
roles of the evaluation and 
the agency? a confusion that 
has on occasion created 
small problems* The most 
salient of these are 

(a) action projects some-* 
times using the evaluation 
staff as a conduit for 
issues more properly 
addressed to their Federal 
project officers? and 

(b) action projects some- 
times assuming that .a Fed- 
eral agency concern is an 
evaluation priority as welT* 
Occasionally? this collabo* 
ration has also resulted in 
some resentment when a proj- 
ect officer emphasized the 
importance of cooperating 
with the evaluationt or when 
evaluation problems or 
information was shared with 
a project officer* 

These mi nor p rob 1 ems are 
outweighed by the positive 
contributions of this joint 
approach* The national 
evaluation would not have 
been possible without a 
coordinated approach invol v- 
ing all three divisions of 
OJJOP* Because persons sub** 
mitting proposals for action 
projects under the alterna- 
t i ve educat ion initiative 
did not really expect to be 
eval uated rig or ou sly? and 
because we had to discuss 
touchy i ssues ( such as col 
lecting data about the crim- 
inal behavior of students? 
impl ementi ng evaluati on 
des igns? and monitor ing 
project impl ementat i on 
activities) with action 



agencies? the evaluation 
woul d have been torpedoed 
from the beginning had we 
not had the backing and 
understanding of the Special 
Emphasis Division? which is 
responsibl e for monitoring 
the action projects* 

There are wel 1 -known 
hazards in collaborating 
with a Federal sponsor on 
the evaluation of a program 
in which it has a vested 
interest* One hazard is 
noted by Cronbach and asso- 
ciates (1980? p* ^)? who 
say? "Insofar as i nf ormat i on 
is a source of power? evalu- 
ations carried out to inform 
a pol icy maker have a disen- 
f r anchi s i ng effect ••• Thi s 
may occur when ••only the 
officials know what is going 
on**' We hope to minimize 
the danger in this area in 
the present evaluation* By 
des i gn? thi s evaluati on i s 
intended to foster the 
devel opment of mor e ef f ec- 
ti ve projects by di recti y 
invol V ing action project 
managers in conducting the 
evaluation and by feeding 
information back to thosa 
managers as a project devel- 
opment tool * Furthermore? 
the open dissemination of 
evaluation reports is a 
mechanism for informing all 
aud i ences of f i ndi ngs that 
may be a source of power* A 
second hazard is discussed 
by Gottfredson (1978) and by 
Weiss (1975)* Because eval- 
uation takes place in a 
03 1 i t i c a1— c orj t?J^ in wh i c h 
multiple stakeholders are 
competing for the allocation 
of resources? ther e is the 
possibility that evaluation 
may be misused in policy 
debates* Although this dan- 
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qer may never be completely 
avoided? we aim to «iniMize 
it by open cominunicati on and 
due scientific circumspec* 
tion« We are above all 
scientists? albeit scien- 
tists tinkering in the area 
of social action and social 
policy* Therefore we seek 
to guard against overly 
effusive statements that are 
not based on sound evidence* 
This may disappoint both 
Federal sponsors and action 
agencies? but it is the only 
defensible course* 

Hazards also confront an 
eval uator collaborating with 
project impl ementers* The 
first is akin to the hazards 
of collaboration with a Fed- 
eral sponsor: Every action 
project wants an evaluation 
to make It look good? and 
wants to use evaluation 
results in its political 
struggle for survival* And 
no action project wants an 
evaluation to be used--as is 
so often the case — to Just- 
ify its demise* Therefore 
project implementers are at 
once eager for and afraid of 
evaluation* The second 
hazard is that? through sym- 
pathetic interaction with 
persons earnestly trying to 
do good? the evaiuator may 
contribute to the misuse of 
evaluation* Our approach to 
both of these hazards is to 
acknowledge that they are 
threats? and again to seek 
umbrage in scientific skep- 
ticism and open discourse* 

A third hazard may exist* 
CommeiTtdtors have divergent 
views about the proper role 
of an evaiuator in influenc* 



ing project process. One 
view holds that it is not 
appropriate to intervene in 
the conduct of a demonstra- 
tion project because such 
evaiuator intervention would 
probably not be available in 
more wide-scale subsequent 
adoptions of a program 
model? thus threatening the 
external val idity (general- 
izability) of the evalua- 
tion* Another v iew hoi ds 
t^at formative evaluation is 
an essential aspect of the 
evaluator*s role? and that 
eval uator s shoul d i ntervene 
by providing inf omat ion 
according to the act ion 
research model * Per 1 of f 
(1979)? yho discusses the 
divergent views using the 
OE/NIE/AIR/ERS exparience 
with the ••Cities in School s^» 
program as an illustration^ 
leans toward the first view* 
We endorse the second? espe- 
cially in the present case* 
The OJJQP prograit is a 
research and develoement 
project* Devel opment of 
models is clearly an appro- 
priate goal? given the state 
of del i nquency prevent i on 
theory and practice at pres- 
ent* The nation's experi- 
ence with delinquency pre- 
vention attempts i s sq. 
fraught with weak programs 
(Qgawa? this volume) that 
excessive worry about evaiu- 
ator intervention leading to 
inappropriate transportabil- 
ity conclusions are prema- 
ture* The primary tasks at 
present are to demonstrate 
tnat soige interventions can 
work and^to learn how to 
facilitate the implementa- 
tion of such ihterventions* 
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Action Project Nrimcs f LocationSf and Award Amounts 



I' Organization Name ) Location I Nunfiber I Amount | Awarded |BeTinning| Ending ( 

* ^ ♦ ♦ ♦ ♦ 

iPrevfjntion of Delinquency through|St. Croixt Virgin |30JSAX0030| $267»312| 8/13/80 | 9/1/80 | 8/31/82 | 

I Alternative Education j Islands I | I ) I I 



I Indi V i dual i eed Intejrated Alter- |Playa Ponce? Puer to j 80J SAX0031 1 S692 f60'5| 8/I^»/80 | 9/1/80 | 8/31/82 | 
) native Education (Otro Camino)| Rico I I I I 1 I 



IProject PREP | South Hronxt Hcm j 80 J SAX0032 | $ I » 1 96 f 967 ( 8/1^/80 | 9/1/80 | 8/31/82 | 

I I York I I I I i t 



iCompton Action Cent^jr — Youtn iComptont Cal i f or n i a | 30J S AXO033 j $607f682| 8/15/80 j 9/1/80 j 8/31/92 | 

I Development Alternative School! I I lilt 



IPeer Culture Development (Chicagot Illinois |Q0/SAXOC3^| $6n6tl9^»| 8/15/30 | 9/1/80 | 8/31/82 | 



iJazzmobile Alternative Education IHarlemt New York |80JSAX0035( S668f0l9| 8/15/80 | 9/1/80 | 8/31/82 | 
I Arts Program | I t I I I I 



iGeorge !• Sanchez Alternative (Houston? Texas |80JSAX0036| S529t583| 8/15/80 | 9/1/80 j 8/31/82 | 

I Education Program | I I I I I I 



(Prevention of Delinquency thr ou yh | Ka 1 amazoo t Michi gan | 80JSAX0037| $268t315| 8/15/80 | 9/1/80 | 8/31/82 | 
) Alternative Education I I ) I t I I 



iPositivt! Action through Holistic |Charlestont South |eOJSAX0038| $883t508| 8/15/80 | 9/1/80 | 8/31/82 | 
I Education I Carolina I I I | | | 



lAlternative Education Program |Haywordt Wisconsin |80JSAXC0391 »539f773! 9/10/80 | 9/1/80 | 8/31/82 | 



(Project RETAIN (Chicago, Illinois | 80J SAX004 3 | H ,08 8 f 98 3 | 9/10/80 | 9/1/80 | 8/31/S2 | 



(Student T»-^3ining Alternatives (Pasadena, Califor- (80JSAX90441 $5v-'?902| 9/10/80 | 9/1/80 ( 8/31/82 ] 
1 through Urban Strategies I nia ( ( I I t 1 



(Prevention of Delinquency j Sewel 1 ♦ New Jersey 181JSAX0012( $602,601(12/16/80 (12/15/80 (12/1^/82 | 
I through Alternative Education | ( | I t I I 
♦ — ». ♦ ♦• ♦ ♦ ♦ ♦ 

(."iilwaukee Youth Ei^ployment Center 1 M i 1 waukee, Wiscon- ( 8 IJ S A XOOl ^ I $ 1 1 1 56 « 105 I 12/16/80 | 12/1/80 (11/30/82 | 
I (son I I I I I I 



♦ ♦ — — ♦ - — • 

(Plymouth-Canton Alternative Pro- } P 1 y mo uth-Canton , |81JSAX0013! 1^*62,77 9(12/16/80 | 12/1/80 (11/30/82 | 
( grams I Michigan ( | I I 1 I 



(Academy for Community Education |*^iamif Florida |91JSAX0015( 77, *^40 ( 12/1 6/80 j 12/1/80 (11/30/82 ( 



(Project Together (St. Paul, Michigan (31JS^X0022( $300,765( 1/19/81 ( 1/1/81 ( 8/31/82 | 
♦ ■ ♦• — ♦ — — 
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Table Z 



Or janUntlonal Characteristics of Alternative Education Action Projects 





{ Location 


1 Type 


1 Primary Mission 1 


tVirgin Islands Department of 
t Education? Elena Christian 


{St. CroiKt Virgin Islands 
1 


{Public school system 

1 

1 


(Education 1 
1 • 
1 » 


tOispens^rio San Antoniot Tnc* 


IPlaya Poacet Puerto Rico 


{ Not-for-profit service 
1 


( development 1 




{South Bronxf New York 


(Public school system 


( Education t 


IJoint Center for Community jCowptont California 
t Studiest Compton Action | 


(Not-for-profit service 
( organization 

i 


( social sarvica 1 
( • 


tPeer CultUv^ C-<^vel opmentt Inc» 


tChicagot 1 1 1 inois 


{ Not-for-profit service 
( organization 


(Youth davalopment i 
1 * 


tJazzmobile* Inc« 


|Harlem» New York 
1 


1 Not-for-profit serv ice 
( organization 


(Arts education 

1 • 


t Association for the Advancement 1 Houston* Texas 


(Not-for-profit service 
( organization 


(Social service t 
1 ' 


|W«?stern Michigan University* 


(Kalama/oot Michigan 


(Stateuniverslty 
( 


1 Post— seconda r y education and I 
( research 1 


{Charleston County School 


(Charleston* South Carolina 


(Publ ic school system 
( 


1 • 




(Heywardt Wisconsin 


1 Tr i bal counc i 1 


( T r i bal gover niiien t 1 




{Chicago* 1 1 1 inois 


{Public school system 


(Education 1 


tConst i tutional Rights 


( Pasadena* Cal i f orni a 


(Not-for-profit service 
I organization 


( ' 


lEducational Improvement Center 


-{Sewel 1 • New Jersey 


(State government sanctioned 
( educati onal service 


1 • 


{Jewish Vocational Services 


{Mil waukeet Wi scons i n 


{ Not- for -prof it service 
{ organization. 


1 J 


1 PI ymouth-Canton Community 


( PI ymouth-Canton* Michl gan 


{Public school system 
1 


(Education 1 
1 • 


{Institute for Innovative 


|Mlami« Florida 


(Not-for-profit service 
( organization 


( • 






(Public school system 
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Record of Accomplishment 

Preventing Delinquency: The Racord of Accomplishment 



Deborah K« Qgawa 



Many del inquency preven- 
tion programs have been ini- 
tiatedt but few have been 
carefully evaluated* It is 
not known whether most of 
these programs have posi- 
tive? nullt or negative 
effects* Among the few pro- 
grams that have been evalu- 
ated reasonably carefully? 
there have been some posi- 
tive results* Examples 
include Alexander and Par- 
sons* (1973) short-term 
behavioral intervention with 
delinquents and their fami- 
lies? Reid and Patterson's 
(1976) attempts to reduce 
aggression and stealing 
behaviors by modifying the 
reinforcement pattern within 
the family setting? and 
Barth's (1979) review of 24 
studies utilizing home-based 
re i nTorcement to alter 
behaviors in the school set- 
ting* The Alexander and 
Parsons study and the Reid 
and Patterson work adhered 
to rigorous evaluation 
designs by using a random- 
ized control group and a 
matched control group? 
respectively* They are also 
exemplary because of the 
clear plausibility of the 
intervention models* 

The^Record 

These examplarV projects 
are rare? indeed* Dixon and 
Wright (1975) reviewed 95 
del inquency prevention 
reports published after 1965 
and concluded that there is 
a paucity of evidence about 



the ef f ecti venesss of 
existing programs? and that 
when evaluations have been 
C3nducted? few projects have 
shown positive significant 
results* They attributed 
part of the problem to 
unclear project goal s and 
objectives? and to difficul- 
ties encountered in imple- 
ment i ng r i gor ous des i gns and 
in obtaining meaningful 
measurements in a fluid 
action program setting* 
Hawkins and Wall (1980)? in 
describing an alternative 
education program to reduce 
delinquency in Florida? also 
identified design? measures? 
and data collection and 
anal ysi s procedures as three 
^major problems in evaluation 
researcn* 

Krisberg (1979) reviewed 
16 exploratory delinquency 
prevention programs funded 
by the Office for Juvenile 
Justice and Delinquency Pre- 
vention* After one year of 
operation? only one project 
had been ab 1 e to i inpl emen t 
even a quasi-experimental 
design* Most of the proj- 
ects could not be evaluated 
in terms of their effective- 
ness because of problems in 
data collection and lack of 
comparabl e control groups • 
In addition? none of the 16 
projects had articulated a 
useful theory about delin- 
quency in their catchment 
areas or had delineated the 
ways in which their services 
woul d reduce the problem* 
Goals were often too ambi- 
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tious or too amdiguousf and 
were not clearly related to 
the problems the projects 
were to address* Krisberg 
also concluded that the 
failure to achieve these 
goals was due in part to 
incomplete planning* 

The story is the same 
everywhere one looks • Wal 1 t 
Hawkinsf Lishnert and Fraser 
(1980) reviewed 36 ••model •* 
Juvenile delinquency preven- 
tion programs? only two of 
which utilized a rigorous 
evaluation design* Although 
many of the programs sug- 
gested positive effects? the 
designs were not rigorous 
enough to exclude other 
rival hypotheses about the 
reasons for the results* 
Janvier? Guthmann? and Cata- 
lano (1980) rated 52 evalua- 
tions of drug abuse preven- 
tion programs for youtfv-on 
the basis of their meti^odo- 
logical rigor* In fewer 
than half (46%) of the pro- 
grams would the evaluation 
designs allow conclusions to be 
drawn* In addition? only 
half of the evaluations used 
at least one outcome measure 
related to drug abuse* Of 
the 52 projects? only 9 had 
an adequate design and at 
least one outcome measiure _.,.. 
related to drug use* 

Terpstra (In press) 
reviewed 52 articles pub- 
lished between 1965-1980 on 
the evaluation organiza- 
tional development efforts 
that involved the collection 
and analysis of quantitative 
data* Reports were rated on 
six dimensions: 

(a) sampling strategy? 

(b) sample size? (c) control 
group util ization? (d) use 
of random assignment? 



(e) measurement strategy? 
and (f) significance level* 
Results show that 5* of the 
studies indicated a negative 
significant effect; 23%? d 
mixed or nonsignificant 
effect; and 67%? a positive 
significant change* 
Although over half of the 
articles indicated a signi- 
ficant positive effect? 
there exists an inverse 
relationship between the 
degree of methodological 
rigor of the evaluation and 
the degree of successful 
outcome; i*e*? studies pur- 
portedly showing positive 
effects were generally lower 
in methodological rigor* 
These deficiencies in metho- 
dological rigor make the 
conclusions about the util- 
ity or effectiveness of the 
interventions questionable* 
Xhe Terpstra analyses are 
valuable because they illus- 
trate the potential for mis- 
guided enthusiasm about a 
project^s perceived effec- 
tiveness in the aosence of 
careful study* 

In sjmmary? many previous 
evaluations of delinquency 
prevention and organiza- 
tional change programs do 
not yield dependable conclu- 
sions about the orograms* 
effectiveness* Design flaws 
are one factor limiting the 
dependability of a study^s 
conclusion* The use of 
irrelevant measures poses a 
second problem in delin- 
quenc y preventi on evalua- 
tions* Some evaluations do 
not include any delinquency 
measures at, all* In addi- 
tion? measures that are used 
are often poorly operation- 
ally defined? and there is a 
dependence on only one 
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source of data? usually 
official records* More mul- 
tiple-measure evaluations — 
^evaluati ons that include 
sel f -reported del i nquency 
measures to supplement offi- 
cial records— are needed to 
reduce the ambiguity of 
evaluation results (cf. Haw- 
kins L Wall f 1980)« 

ItlggLY^i n_ E valua tion 

Another majoV problem 
with delinquency prevention 
evaluations has been that 
many of the previously eval- 
uated programs have not 
implemented truly plausible 
i nter vent ions based on a 
theory (cf* Glasert 1980)* 
Programs often fail to arti- 
culate a theory of delin- 
quency prevention that would 
provide a conceptual frame- 
work for project planningt 
implementat ionf and evalua- 
tion* Consequently f evalua- 
tors have to ferret out? 
post hocf underlying theo- 
retical assumptions* Pro- 
grams that do not utilize 
theory add little to the 
development of knowledge in 
the area of delinquency pre- 
vent! on? and implementation 
often suffers because proj- 
ect implements rs have no 
standard against which to 
assess their interventions* 

Due to the weaknesses and 
limitations of past evalua- 
tion research in the area of 
del i nquency f i nnovat i ve 
approaches are needed* One 
promising approach is the 
action research model 
(Lewinf 1947)* Action 
research is the study of 
actions as a method for 
advancing both knowledge and 
practice? through a cycle of 
problem analysist planningt 
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execution? evaluation? and 
replanning (Sanfordt 1970)* 
The first step in the action 
research model is planning? 
which involves defining the 
problem and then examining 
ways to resolve it in rela- 
tion to the available 
resources* Once an overall 
plan has been formulated? 
the next stags? execution? 
begins* A fact-finding step 
then evaluates the action 
that has been executed* 
This evaluation provides an 
opportunity to gain new 
insights about the plants 
strengths and weaknesses and 
serves as a basis for the 
next step? modification of 
the plan* Action research 
thus involves a spiral of 
steps: It allows continuous 
i mprovement of a program 
through evaluation of the 
results of each action to 
provide a rational basis for 
planning the next action* 
In order to gain insight 
into a process? change roust 
bs created? followed by 
observation of the new 
effects and dynamics* As a 
result of this cycle of 
activities? s rogr ans shojl d 
become more effective* 

Although this process was 
widely used in the 1940»s by 
the Research Center for 
Group Dynamics and the Com- 
mission on Community Inter- 
actions? action research has 
never been influential in 
psychology or the social 
sciences in general (San- 
ford? 1970)* Very recently 
there has been a small 
resurgence in the use of the 
action research model (Len- 
row? 1970; Grant ^ Grant? 
1970; Hoff? 1970)* Scriven 
(1967)? in distinguishing 
between formative and 



suflimative evaluationst began 
to approximate the Lewinian 
model of action research. 
Recent writing on the evalu- 
ation of delinquency preven- 
tion efforts is now moving 
in this direction* Hawkins 
and Hall 11980) delineate 
standards for evaluating 
delinquency prevention pro- 
grams that include not only 
a summativet or outcome? 
component? but a formativet 
or process monitoring? com- 
ponent as well# Summative 
evaluation involves rigorous 
research designs? standard- 
ized measures? and an appro- 
priate research time frame 
which provides for longitu- 
dinal follow-ups# Process 
monitoring describes the 
program? making replication 
possible* This description 
includes the context of the 
program and the selection of 
participants? and it 
includes the documentation 
of intervention strategies* 

E ya\ uator-I mgl ementer Cglja- 

Krisberg (1979)? Oixon 
and Wright (1975)? Janvier 
et al# (1980)? and Glaser 
(1980) all advocate the use 
of theory in guiding program 
development and evaluation* 
The greater the degree of 
specificity of the theory? 
the more readily identifia- 
ble are a program's set of 
measurable goals* These 
goals then should provide 
the framework around which 
intervention strategies are 
to be tailorec^o In addi- 
tion? project implementers 
should be involved as col la- 
bor ator s in research* 
Again? ir accordance with 
the Lewinian action research 
model? involvement is an 
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important aspect of group 
dec is ion- ma king since it 
minimizes resistance to con- 
sidering the problems and 
possibilities of an objec- 
tive and it allows expres- 
sion of several alterna- 
tives* Thus* involvement of 
orogram implementers in the 
research process may reduce 
the program staff's resist- 
ance to, and anxiety about* 
being evaluated. More 
important, the underlying 
assumption is that project 
staff members are more 
effective change agents if 
they participate in and have 
a stake in the decision-mak- 
ing and research process. 

&223-ii§!nBlSS. 

Many of these recommenda- 
tions have already been 
incorporated in evaluation 
studies. Empey and Lubek 

(1971) and Empey and Erikson 

(1972) have integrated 
sociological theory in 
delinquency prevention 
intervention efforts. Their 
work included a formulation 
of the theory, intervention 
strategies* and methods for 
empirically testing these 
formulations. Alexander and 
Parsons (1973)* recognizing 
the paucity of demonstrable 
intervention effects in the 
psychotherapy literature, 
incorporated a strategy in 
their evaluation of family 
therapies which involved 
four main goals: 
(3) presentation of a clear 
description of intervention 
techniques, (b) process 
evaluation, i.e., describing 
and evaluating the beha- 
vioral changes in family 
process expected from the 
intervention, (c) summative 
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evaluationt i«e«f using 
clearly defined and nonreac* 
tive behavioral criteria to 
evaluate the effectiveness 
of the intervention? and 
(d) incorporation of con- 
trols for maturation and 
prof ess i onal attention (p« 
219) • This study was able 
to effectively utilize a 
stringent experimental 
design with three groups: 
families receiving treat- 
ment? famil ies receiving 
a1 ter native forms of famil y 
therapy? and families 
receiving no professional 
treatment* This study 
implemented a strong design 
and demonstrated positive 
treatment effects in the 
reduction of recidivism in 
deliquent teenagers* 



The School Action Effec* 
tiveness Study is an attempt 
to use the experience of 
previous programs and the i r 
evaluations to anticipate 
and avoid as many pitfalls 
as possible* It aims to 
assist in clarifying goals 
and theory? and their link- 
ages with short-term or 
i ntermediary object ives and 
the interventions aimed at 
bringing these objectives 
about* And it '1 so aims to 
provide work^^^.e structures 
for managing project imple- 
mentation and eval uat ion 
according to the action 
research model * 1 1 combt nes 
f or mati ve evaluat i on or 
project development with 
r igorous eval uat i on* 
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Making Inferences about Project Effectiveness 



Once a project has imple- 
mented some plausible inter- 
vention intended to influ- 
ence student attitudes? 
behavior? or development? 
assessing the consequences 
of that Intervention becomes 
important* Making this 
assessment is not always 
easy* Young people are 
growing and changing all the 
time* Rates of participa- 
tion in delinquent behavior 
apparently rise and then 
fall with age* Scholastic 
competencies usually grow 
over time? but at different 
rates for different peopla* 
Students make new friends 
and abandon old ones? and 
every parent knows that his 
or her child^s tendency to 
conform or rebel is differ- 
ent at different stages of 
development* Isolating the 
influence of some specific 
experience? intervention? or 
set of interventions is 
therefore difficult* 

Making inferences about 
the causes of some differ- 
ence in student outcomes — a- 
bout the effects of planned 
interventions— i St however? 
a major goal of evaluation* 
Put another way? an aim of a 
thorough evaluation is to 
determine whether an 
observed difference in stu- 
dent behavior or attitudes 
( if any difference is 
observed at all) can 



I am grateful for comments 
by Deborah Daniels? Denise 
C* Gottfredson? and Jane St* 
John on a draft of this 
chapter* 



reasonably be attributed to 
a specified intervention* 
Certain conditions make the 
saarch for the effects of an 
intervention easier; other 
conditions preclude making 
any confident inferences* 
Those conditions are the 
topic of this chapter* An 
excellent discussion related 
to this topic exists else- 
where (Cook t Campbell? 
1979)? and readers may wish 
to see that source for an 
elaboration of sorwe of the 
points made here* 

When an educational or 
other intervention has been 
executed with fidelity? the 
evaluation task focuses on 
learning the consequences of 
that intervention* In prac- 
tice? of course? no project 
can wait until after the 
intervention has oeen imple- 
mented to begin work on this 
task* Conditions must be 
established at the outset to 
allow the conclusion that 
ooserved outcomes were 
brought about by by the 
intervention? ratner than by 
something else happening at 
the same time? or by the 
natural course of develop- 
ment* ^Project implementers 
aid evaluators ignore the 
establishment of these con- 
ditions at their peril; 
causal interpretations of 
observed outcomes are dubi- 
ous unless rival explana- 
tions can be ruled out* 

Suppose? for example? 
that an alternative educa- 
tion project involving group 
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and individual counseling 
Mere to be implemented* The 
counselors implementing the 
treatment believe that only 
students willing to partici* 
pate fully and amend their 
previous conduct are amena-* 
ble to this treatment«<l> 
Thereforet only students who 
express an earnest willing- 
ness to commit themselves to 
the project become involved 
in the counsel i ng activi* 
ties« Under these circum* 
stances f counselors often 
make claims for the effec* 
ttveness of their intervene 
tion by comparing the past 
behavior of these students 
with their behavior during 
or shortly after counseling* 
Ort they may claim effec- 
tiveness based on a compari- 
son of students receiving 
treatment with apparently 
similar students who did not 
become involved in treat- 
ment* These claims are on 
shaky, ground* Any differ- 
ences may be due to the 
desire of the individuals 
involved to reform? or to 
maturation? and may have 
nothing to do with the 
treatment* The rival expla- 
nations are as good as the 
one the counselors wish to 
make* 

Consider a second exam- 
ple* Educators are conduct- 
ing a project involving 
individual i zed education 
plans developed by a spe- 
cialist in collaboration 
wi th their students* regular 
classroom teachers* The 
basic idea is to make a 
diagnosis of each student^s 
needs and specify achievable 
academic and behavi oral 
objectives; the specialist 
i s to serve as a kind of 
ombudsman to promote the 



educational welfare of the 
students* Classroom teach- 
ers are asked to refer to 
the project students for 
whom these special services 
seem appropriate? and they 
are given a list of criteria 
to guide them in making 
referrals* Referral cri- 
teria call for students who 
exhibit mild behayi or prob- 
lems? sjch as difficulty in 
impulse control or persis- 
tent truancy? or for stu- 
dents whose c 1 ass work per- 
formance is at a level below 
the teachers^ expectations* 
The educators admi n ister 
tests (and collect certain 
other information) pr ior to? 
dur i ng? and after students • 
i nvol vement in the indi vi du- 
al ized education* Because 
scores on these tests 
i ncrease? the educators 
claim that the treatment i s 
ef f ecti ve* But tha se claims 
are quest i onabl e* Scores on 
educati onal tests al most 
always go up over time? 
espec ial 1 y when any i nstruc- 
tion is occurring* There- 
fore? gains cannot necessar- 
ily be attributed to the 
treatment* 

Suppose? then? that a 
novice educational 
researcher wanted to examine 
this same project more 
closely* The novice tries 
to construct a ••control*' 
group using students of the 
same race? sex? grade? and 
age as the students involved 
in the individual education 
treatment? drawn from the 
same classrooms as the stu- 
dents referred to the proj- 
ect? i*e*? by ••matchi ng*** 
Fortunate! y? the same tests 
administered tc tfie project 
clients were also adminis- 
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tered to students not 
involved in the projects 
Behold? the students receiv- 
ing individualized attention 
show smaller gains than the 
'•control" students^ The 
novice concludes that the 
treatment was actually Harm- 
ful Istudents would have 
learned more if not involved 
in the project). Perhaps 
not# Recall that the stu- 
dents referred to the treat- 
ment were performing below 
expectations? were exhibit- 
ing behavior problems? and 
were often truant* These 
students may be expected to 
show educational growth that 
was slower than the growth 
of other ••matched'* students 
in the same class in the 
absence of any special 
intervention. Thus? differ- 
ence in expected educational 
growth rates is an explana- 
tion with as much credibil- 
ity as that of the novice 
educational researcher* 

Sophisticated measurement 
and statistical techniques 
can sometimes help sort out 
the evidence about an inter- 
vention's effects under dif- 
ficult conditions* But 
these non-experimental 
efforts to make Inferences 
are plagued with uncer-< 
tainty* Few such efforts 
that capture the attention 
of other methodol ogists go 
unchallenged for long* In 
short? positive steps to 
assure an intervention's 
evaluatabil i ty are essential 
if confident statements are 
to be made about the proj- 
ect's effectiveness In terms 
of its intended outcomes* 



Randomization^and.Al terna- 
ti ves 

A riumber of methods allow 
reasonably confident infer- 
ences* Of these the true 
experiment and some quasi- 
experimental methods such as 
single-subject or ABA 
designs? and regression dis- 
continuity designs? require 
some degree of experimental 
control over %he timing of 
treatment* Other quasi -ex- 
perimental methods? such as 
i nter rupted X ime-s er ies 
designs? require a large 
number of observations over 
a long period of time? 
tDgether with clear-cut 
changes in some environmen- 
tal influence at a particu- 
lar point in time* (These 
methods and others are 
described in Cook C Camp- 
bell ♦ 1979*) It is always 
wi^e to consider whether any 
of the various quasi-experi- 
mental designs are likely to 
be credible in ruling out 
rival explanations of a par- 
ticular outcome*<2> Truci 
experiments are usually 
administratively simpler? 
fraught with fewer technical 
difficulties? and more gen- 
erally understood* 

True experiments usually 
Involve the random assign- 
ment of Individuals? class- 
rooms? schools? time peri- 
ods? or other units to 
alternative treatments* No 
single procedure is always 
effective In guarding 
against all rival interpre- 
tations? but randomization 
Is a helpful general purpose 
mechanlsm*<3> When two or 
more groups are created 
through randomization? they 
are equivalent within the 
limits of random sampling 
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error; methods for estimat- 
ing the size of this error 
are available* Had true 
experiments been conducted 
in the hypothetical cases 
described earlier? confident 
conclusions would have been 
poss ib1e^*provided that cer- 
tain other conditions neces*-* 
sary for inference were also 
present* 

Other Con di tions^Hak^ng for 
Conf i d ence 

To make confident 
interpretations of evalua* 
tion results? three impor- 
tant additional conditions 
are required? adequate sta- 
tistical gowert sound meas- 
ureRient of the outcomes of 
interest? and comglete 
|n forma tio q • 

^ Statements about treat*- 
ment effects made by evalua- 
tors and other scientists 
are probabilistic* Gener- 
ally? the degree of confi- 
dence one may have in a con- 
clusion is expressed by 
indicating the probability 
of this outcome occurring by 
chance if the intervention 
were completely ineffective* 
Thi s is what stati st ic ians 
mean by "^signif icance*** A 
significant result is one 
that is unlikely to occur by 
chance* Many scienti sts ? as 
wel 1 as lay persons ? are 
confused by the distinction 
between the size of a dif- 
ference and its statistical 
significance* Large differ- 
ences in the average delin- 
quency rates or educational 
achievement test scores 
between two groups can be 
nonsignificant* And? small 
differences can be signifi- 
cant* In most delinquency 



prevent i on interventions? as 
indeed in most educational 
i nter vent ions? treatment 
effects are likely to be 
small* Detecting such small 
differences with confi- 
dences—and understanding the 
paradox of nons ignif icant 
large di f f erences--requi res 
a consideration of statisti- 
cal power* 

Power is the probability 
of detecting a difference of 
any given size that hypo- 
thiatically exists* The most 
i mportant pr inc ipl a i nvol ved 
is that the probability of 
detecting a true difference 
with conventional signifi- 
cance tests increases as the 
number of experimental units 
(students? cl asses? time 
per iodSf or school s ) 
increases* If the true dif- 
ference is large? a smaller 
number of exoer i mental uni ts 
is required to detect it 
with a given level of prob- 
aoility* Thjs? when small 
treatment effects are 
1 i kel y? 1 arge numbers of 
people must be given the 
treatment to make a signifi- 
cant result probable* when 
effect sizes are large? 
smaller samples can be used 
and stil 1 provide a reason- 
able probability of detect- 
ing the effects* In most 
delinquency prevention eval- 
uations? large sample sizes 
are needed to gain statisti- 
cal power bee ause i nterven- 
tion effects are likely to 
be smal 1 * 

The s econd add i t i onal 
condition necessary for mak- 
ing confident statements 
about an interventi on* s 
effects is the sound meas- 
urement of the outcomes that 
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may be influenced by the 
intervention. The review by 
Otgawa (this volume) t as well 
as our experience in trying 
to obtain sound measures of 
academic achievement? delin- 
quent behavior? and other 
potential outcomes in this 
evaluation? implies that 
this condition can often be 
difficult to meet# There is 
no way to confidently con- 
clude that an intervention 
prevents delinquency when no 
good measure of delinquent 
behavior is available* 
Measurement Issues are dis- 
cussed more thoroughly in 
another chapter* 

The third necessary con- 
dition? complete informa- 
tion? is also often diffi- 
cult to meet because schoDl 
populations are transient? 
and because students in high 
risk of delinquent involve- 
ment are often truant or 
drop out of school alto- 
gether* In addition? some 
students for a variety of 
reasons never receive the 
intended treatment in full 
form* The necessity of 
obtaining outcome measures 
for these individuals is 
frequently overl ooked* 
Attrition weakens an evalua- 
tion by effectively diluting 
the treatment* And? if 
information for some indivi- 
duals is not available? a 
number of equally plausible 
rival explanations for out- 
comes may exist? thwarting 
confident interpretation* 



Soine^Common^Qbiections 

Educational practi ti on- 
ers? counselors? and social 
service workers often object 
to establishing tne condi- 
tions necessary for making 
confident inferences about 
the services they provide or 
about their organi rational 
change efforts* These 
objections take many forms: 

I. "I know this inter- 
vention to be effective? and 
therefore evaluation is 
unnecessary*" 

2« ••Randomization is 
uneth ical *•• 

3. ••Asking students to 
report about their behavior 
or school is demeaning*'^ 

4« ••Evaluation is too 
much work; it detracts from 
other programmatic efforts? 
or makes the intervention 
difficult to implement as 
intended*** 

5* ••Evaluation threatens 
the staoility of the project 
by creating problems for its 
manager or powerful others 
in the manager's environ- 
mant*** 

6* ••Evaluation restricts 
the project's freedom of 
action in devel oomental 
stages*" 

§X§lU3tion_Is JJnnecessarjj 

••I know this treatment 
works* Evaluation is not 
necessary*" 

No intervention involved 
in the Alternative Education 
Program has been demon- 




Inferences 

strated to be effective in 
preventing del inquency« 
Most have not been demon- 
strated to be effective in 
achieving any of the other 
program goa1s« Some inter-* 
vent ions being contemplated? 
e»g«f the Student^-Team- 
Learning approach being con- 
sidered in Charleston? have 
reasonably been shown effec- 
tive in increasing student 
satisfaction? learning? and 
positive peer relations in 
implementations conducted by 
skilled researchers* Repi i-* 
cations under more typical 
conditions are lacking* 
Others? such as the FOCUS 
approach? have been sub- 
jected only to unreplicated 
examination involving a lim- 
ited range of potential out- 
comes* Still others? such 
as PLATO and a host of other 
interventions? have never to 
the best of our knowledge 
been satisfactorily evalu- 
ated at all* (We acknow- 
ledge that our standards for 
a satisfactory evaluation 
are considerably higher than 
the standards of those who 
market or otherwise dissemi- 
nate these products*) Even 
were it true that some eval- 
uation had found an inter- 
vention to be effective in 
preventing delinquency? the 
repl icabi I i ty of that inter- 
vention and its results 
would be an important evalu- 
ation question in a demon- 
stration program* Evalua- 
tion is therefore necessary* 

Randomiz at i on Is Unet hic al 

"Refusing or delaying 
this service to allow for 
its evaluation is unethi- 
cal. 
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The denial of some ser- 
vice? known to be effective? 
for evaluation purposes when 
the aggregate harm to the 
i ndi vidua 1 outwei ghs the 
resulting aggregate benefit 
is unethical* This is a 
fundamental tenet of utili- 
tarian ethics* Rule ethics 
arguably imply further that 
^denial of a service? known 
to be beneficial to an indi- 
vidual? may be unethical 
regardless of the aggregate 
harm or benefit* But the 
denial of a service when its 
efficacy is unknown is not 
unethical * Indeed? when 
effects are unknown? the 
ethicality of administering 
the treatment is doubtful? 
especially if the treatment 
is not under serious eval ua- 
t i on* 

When the effectiveness of 
a treatment is unknown? the 
alternative to experimenting 
with people is fooling 
around with people* Con- 
sider again some examples* 
Peer or group counseling is 
a plausible intervention 
because it recognizes the 
powerful influence of peers 
on a student^s behavior* 
Some reluctance to assign 
students randoml y to thi s 
kind of counseling has been 
encountered* This resist- 
ance is based in part on 
concerns about denying a 
needed service to individu- 
als who would be randomly 
assigned to a waiting list 
or control group* The 
excellent background paper 
for the Alternative Educa- 
tion Program Announcement 
(□ffice of Juvenile Justice 
and Del i nquency Prevention? 
1980? Appendi x 3) makes 
clear? however? that the 
appropriate approach to peer 
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counselling interventions is 
an experimental one* "Given 
the growing popularity of 
peer counsel ing and the 
likelihood that some alter- 
native programs will use it? 
it is essential to rigor- 
ously assess its effects in 
alternative education pro- 
grams* I t^cannot^be^assuined 
that PQ sTtlyilresults^iiill 
be fou nd^'^Tp* 24? emphasis 
added) • 

The ethical approach to 
such interventions is to 
eval uate them* As the Amer- 
ican Psychological Associa- 
tion's Task Force on Evalua- 
tion and Accountability 
(1978) put itf ••In the vast 
majority of cases the only 
really ethical position lies 
in providing the public with 
effective services or ser- 
vices whose effectiveness is 
under systematic evaluation** 
(p* 305)* 

In a second example? 
alternative schools may seek 
to keep dropouts or poten- 
tial dropouts in school • 
Again? resistance to random- 
ization has been based in 
part on concerns about deny- 
ing a needed service to 
individuals who would ran- 
domly be assigned tr a con- 
trol group* Yet thj bene- 
fits of continued schooling 
for youths with high dropout 
potential have not been dem- 
onstrated. Del inquent 
behavior typically falls off 
when youths drop out of 
school and may even be lower 
dur i ng summer recess from 
school (GDttf redsont 1931) • 
Some delinquency refi^earcher s 
have commented that ••dropout 
is a satisfactory solution** 
for some youths (Elliott? 



1966) f and that we should 
rethink the appropriateness 
of trying to keep youths in 
school as long as possible 
(Glasert 1975^ p# *7)# 
Bdchman? Greent and Wi rtanen 
(1971) characterize dropoutf 
as a symptom of more basic 
organizational problems? 
rather than as a problem for 
the individuals who drop out 
themselves* Treating this 
symptom rather than the 
underlying problems ••may in 
this instance do more harm 
than good for two reasons • 
Firstf the treatment has 
some unpleasant side effects 
. . . • Secondf treating the 
symptom may distract us from 
the more basic problems'^ (p« 
179)* 

The importance of learn- 
ing the ef f ects--which may 
be positive? negative? or 
mixed~-of keeping potential 
dropouts in school cannot be 
overestimated* Thi s is an 
issue with tremendous policy 
relevance* The ethical 
route lies in the rigorous 
evaluation of any program 
designed to keep potential 
dropouts in school* 

The case for randomiza- 
tion when the effectiveness 
of an intervention is unk- 
nown has been elaborated 
elsewhere (Boruch? 1975; 
Gottf redson? 1973) • Random- 
ization is often considered 
3 model of fairness in allo- 
cating benefits or risks 
(Feinberg? 1971) arid has 
much to recommend it on that 
basis alone when the conse- 
quences of a treatment are 
unknown* Surely the once 
commonly accepted and widely 
practiced medical treatment 
involving blood letting 
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Mould have been abandoned 
soonert saving countless 
MveSf had anyone performed 
the necessary evaluation 
{cfm Eisenbergt 1977)« 

Asking Certain ques^Lons Jg 
Deme an i nq 

••Why should innocent ^ 
youngsters be asked If they 
have committed crimes? VThy 
should they be asked if 
school stinks? Isn't this 
deroeaning?** 

Ouest^^^ be 
asked of students to learn 
how interventions affect 
them* Student self-reports 
on their conduct are one? 
albeit imperfect? method of 
learning about their delin- 
quent behavior* Self-report 
measures are well studied 
(Hindelangt Hirschit L Weist 
1931)t anJ we know they have 
jseful degrees of validity 
for an evaluation* Stu- 
dents' perceptions of their 
schools and their experi- 
ences in schools are impor- 
tant in assessing school 
cl imate for diagnostic pur- 
poses? for character izi ng 
the school environment? for 
assessing project effective- 
ness* And because attitudes 
are important variables that 
theory impl ies mediate 
between pi aus ibi e interven- 
tions and delinquency or 
dropoutf measures of these 
attitudes are also impor- 
tant* 

Asking these questions is 
appar entl y not demean i ng* 
The overwhelming majority of 
students (81*3%) who com- 
pleted the School Action 
Effectiveness Student Ques- 
tionnaire reported that it 
was ver/ or somewhat 



i nterest i ng* Discussions 
wi th students i mpl y that 
they general 1 y apprec iate 
being asked their opinions* 
Ignoring the messages stu- 
dents give us when they 
answer these questions may 
be demeaning; asking the 
questions is not* 

ivaXuat|on_Gets_in_the_Wa^ 

**We have not budgeted for 
a staffer to collect those 
dat£}* Spending effort on 
developing the evaluation 
takes precious time away 
from getting our work done* 
We cannot meet our quota for 
persons served if we have to 
establish a control group 
too*** 

These are real problems* 
Hore than a decade ago a 
prison warden (Parkt 1965) 
told a story that is modi- 
fied slightly and repeated 
below:«^> 

Once upon a time 
there was an alternative 
education project direc- 
tor who was riding a 
tiger* By holding on 
with both hands and 
struggling very hardt he 
could steer the tiger 
just the slightest bit* 
The project director's 
best efforts were not 
enough to prevent the 
tiger from taking an 
occasional nip out of 
his leg* 

One day an evaluation 
researcher visited and 
saldf **1 see you are 
riding a tigert** thus 
demonstrating her keen 
insight into the project 
director's pi ight* At 
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that momentf the tiger 
took another bite from 
, the director's leg* 

The researcher 
observed solemnVy? and 
* presently issued a find- 

i ng: "You know? that 
tiger is biting yout and 
seems to enjoy it* 
Someday he will eat you 
al 1 gone~unl ess? that 
iSf you avail yourself 
of my services*" 

••You know about rid- 
ing tigers?** 

••MOf" said the 
researchetf ••but I have 
extensively studied 
Siamese kittenSf and I 
am sure the principles 
are the same*^' 

Although he was able 
to devote only a small 
part of his attention to 
the researcher's state- 
mentf being almost fully 
absorbed in efforts to 
control the tiger? the 
project director made an 
execut i ve dec i s ion* 
Since things were going 
badly at the moment t he 
thought he had little to 
lose by getting the 
researcher * s hel p* 
••Finef" he saidf ••come 
aboard and give me a 
hand*" 

••Not so fastf" 
answered the researchert 
••we scientists can't go 
slapdash into things* 
^ In the first placet we 

must develop an evalua- 
tion designf administer 
some ques t i onna i res f and 
develop a data retrieval 
mechanism* And in the 
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second placet we must 
find a quieter tiger* 
Simply impossible to 
properly study such a 
rambunctious creature*" 

••But this is the only 
tiger I have? and if I 
let it go he will run 
around eating a 1 ot of 
i nnocent student s t and 
scare them into dropping 
out of school 

••Boshf^^ replied the 
researcher* "You are 
being rigid* If you 
will loosen your grip a 
littlet we can randomly 
assign it to individual- 
ized training* We'll 
provide it with caring 
and concerned trainers* 
If we find that it eats 
fewer studentst or even 
i f an intervening varia- 
ble is inf 1 uencedf we 
will have a result* Of 
course if it eats more 
studentSf we will still 
have a resul t--ppposi te 
d i rect i on natural 1 y* So 
we can't loset can we? 

••Wellf" said the 
project d irector f who 
was getting confused by 
the researcher's logic? 
"I'm not sure about 
that* Help me get this 
t i ger under control f 
then we can think about 
evaluation*" 

"Nowf nowf" answered 
the researcherf "you 
can*t just start these 
things without prelimi- 
nary study* We must 
devise a management 
information systemt get 
some data from the 
tiger's point of viewf 
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and prepare some 
feedback* Right nOMt we 
cannot even describe 
what you two are doing*" 

••Chompt** said the 
tiger* 

All rightt all 
rightt*' said the project 
di rector t ••descr i be 
already — but hurry*" 

"Nowt 1 et • s see-- I 
Mould say at the 2% 
level- of confidence that 
we could have some pre- 
1 i mi nar y resul ts that 
would point up areas 
requi r ing intens ive 
study in about five 
yearst plus or minus 2*3 
years*" The researcher 
looked pleased* 

"I don't 1 ike to be 
an obstructionistt" 
replied the administra- 
tor ♦ "but I have this 
tiger to cope with nowt 
and I am not sure I will 
be around in 5 plus or 
minus 2*3 years*" 

"You must realizes" 
said the researcher* 
"that we must develop 
criteria* select sub- 
jects* and make sure the 
results are not due to 
the use of catnip rather 
than the tiger's indivi- 
dualized training plan* 
Besidesf we will gener- 
ate valuable information 
to help the next project 
d i rec tor * You woul dn • t 
want the next poor soul 
who comes along to have 
the same problems you 
are having with this 
tigert would you?" 



"^Hmmmmmmmmmt" 
hummmnmmed the pro ject 
d irector f " that doesn't 
sound at al 1 practical* 
However* while you are 
working on the evalua- 
tion design* perhaps you 
can help me shift my 
grip a little* Down a 
ways and a bit to the 
right should do it*" 

"I'm pretty commit- 
ted*" said the 
researcher* "to devel op- 
ing a rigorous national 
evaluation* but I can 
give you a few minutes 
of consul tat t on* Here* 
hold these data collec- 
tion forms and I will 
help you optimize your 
grip*" 

The project director 
reached out for the 
research tools* momen- 
tarily loosening his 
grip* The tiger 
promptly turned and ate 
him al 1 gone* 

The researcher 
regarded the scene 
sadly* "Just when he 
was coming around to my 
frame of reference*" 



The project director* may 
his soul rest in peace* had 
a point --several poi nts* 
But his problems both ante- 
dated and went beyond the 
problems created by the 
researcher* This admi ni s- 
trator was not in control* 
An organization must be sta- 
ble and have enough control 
of its course to be able to 
make sound decisions and 
i nvestigate the consequer^ces 
of its action* even though 
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the crisis may stimulate hir» 
to do something* 

An organization must be 
Milling to dedicate a share 
of its attention and 
resources to inquiry if it 
is to engage in and use 
research— indeedf if it is 
to develop at all* A fail* 
ure to budget adequate 
resources for evaluation is 
a direct indication that an 
organization does not value 
the contribution that evalu- 
ation can make to project 
development. It is a prob- 
lem that must be overcome; 
otherwiset the attempt to 
evaluate must be abandoned. 
Evaluation research does 
require a commitment of 
resources. 

In the disorganized case 
of the project director rid- 
ing a tigert devoting atten- 
tion to evaluation is indeed 
hazardous. Few project 
directors -#ould arguet ho<<- 
evert that the work they 
wish to accomplish is riding 
a tiger. If the project 
director is out for a tiger 
ridet research will get in 
the way. But as Lewin 
(1946) put it: 

In a field that lacks 
objective standards of 
achievement? no learning 
can take place. If we 
cannot judge whether an 
action has led forward 
or backward? if we have 
no criteria for evaluat- 
ing the relation between 
effort and achievement? 
there is nothing to pre- 
vent us from making the 
wrong conclusions and to 
encourage the wrong work 
habits. Real i st ic 



fact-finding and 
evaluation is a prere- 
quisite for any learn- 
ing. Social research 
should be one of the top 
priorities for the prac- 
tical job of improving 
intergroup relations (p. 
35). 

Good managers want to 
learn; they want to promote 
the rational development of 
their activities. Far from 
diverting attention from 
project devel opment ? eval u a- 
tion is a tool of project 
and organizational develop- 
ment (French & Bell? 1978). 
To use this tool* the proj- 
ect must be willing to grow 
and develop? and to devote 
resources to the learning 
enterpr i se. 

Probl ems in impl ementing 
an evaluation often surface 
when a project has diffi- 
culty filling its institu- 
tion or meeting service 
delivery quotas. Any kind 
of control over assign- 
^Q^X, — admissions? selection? 
di f f erent ial treatment— is 
difficult when an organiza- 
tion has trouble at the 
front doDr. If its services 
are not in demand? or if the 
demand is for services of a 
different kind than the 
project aims to provide? 
problems are created for 
evaluation and for project 
implementation. 

For example? teachers may 
refuse to make referrals to 
a treatment unit? or persons 
deemed in need of the ser- 
vices may not avail them- 
selves of it. Thei a prob- 
lem exists with or without 
the evaluation: too few 
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eligible candidates for the 
project* This problem may 
affect only the evaluation 
(which is rarely the cast^t); 
pools of eligibies will not 
be of sufficient size to 
make control over assignment 
possible* In this situa- 
tionf the small number of 
candidates could indicate a 
lack of commitment to self- 
study and evaluation* On 
the other handf the problem 
may affect other aspects of 
implementation* The una* 
vail ability of a pool of 
eligibies may result in the 
provision of services to an 
inappropr i ate group? or the 
project may be unable to 
attract clients truly in 
need of services* In this 
case* a more serious project 
management problem exists* 

Ironically? the failure 
to take stops to evaluate 
rigorously may lead to the 
perpetuation of the problem 
because learning does not 
take place? interventions 
may not be modified to 
become mora appropriate or 
attractive? and the organi- 
zation has only soft evi- 
dence or vague appeals to 
use in its effort to extend 
services to persons deemed 
in need of them* At the 
very least? the unavailabil- 
ity of sufficient pools of 
eligibies in a pilot study 
or demonstration project is 
a major defect in a project 
of this type? because it 
limits what can be learned 
about the effects of the 
project* 



"Elements of the evalua- 
tion procedures thwart the 
project because the organi- 
zational hierarchy* the 
project*s staff? or other 
elements in the project's 
environment resist it; this 
threatens the project's sta- 
bil ity*" 

As Oay (1931) has pointed 
ojtf a project has a higher 
likelihood of being institu- 
tionalized and of creating 
change in the system if cer- 
tain conditions are present* 
Among these are the follow- 
i.ig: (a) Key decisionmakers 
have a reputation for inno- 
vation and experimentation* 

(b) The system is monitoring 
the project and receiving 
information about it* 

(c) The oroject sees itself 
as a demonstration or pilot 
project* 

A static organization? 
one that does not wish to 
create change or one that 
clings to the status quo* is 
naturally wary of rocking 
the boat* Beer (1960) notes 
two major sources of change 
in an organizational system: 
crisis and information* 
Sooner or later? an organi- 
zation that avoids the impe- 
tus to change provided by 
information may be expected 
to experience crisis* 

Or gani zati onal develop- 
ment specialists generally 
bel ieve that the persons 
affected by a project or by 
research on it should be 
involved in the development 
of research questions and 
designs (see e*g*? Frohman? 
Sashkin? C Kavanaugh? 1976)* 
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This involvements which we 
aim to foster in the School 
Action Effectiveness Studyf 
is important not only in 
overcoming resistance to 
evaluation* but also in 
fostering the subsequent 
utilization of information 
generated by the evaluation^ 
4 useful strategy for a 
project director experienc- 
ing staff resistance would 
be to replicate parts of the 
Program Development Evalua- 
tion process with members of 
the project's staff or the 
organization's hierarchy^ 
This approach may encourage 
the members' support for 
activities they would come 
to see as relevant and 
important* 

E vallia t ion _ R £ s trj,c ts _ F re e d o in 

••My project is trying to 
get started • We have enough 
problems i^ithout trying to 
adhere to onerous structures 
created by an eval uat i on*'' 

when a project is going ^ 
through the first months of 
starting its operationt 
activities aimed at discov- 
ering the effectiveness of 
its interventions are usu- 
ally inappropr i ate* Thi s is 
especially true when proce- 
dures have yet to be devel - 
opedf staff members are 
still unclear about project 
goals or perhaps have not 
yet been hired at all t space 
has not been rentedf and no 
services are being rendered* 
At this pointf evaluation 
activity is appropriately 
aimed at clarification of 
project goalSf rationale? 
objectives? plans? history? 
implementation? and setting* 



even at this early stage? 
however? it is essential to 
begin planning for summative 
kinds of evaluation activi- 
ties* This is especially 
true with time-1 imi ted dem- 
onstration projects* If 
early steps to develop a 
framework for evaluation are 
not taken? the project may 
never be evaluated in its 
lifetime* The trick is to 
balance activities so that 
they are appropriate for the 
developmental stage of each 
project at any ppint in 
t i me* 

The possibility exists 
that some projects will 
remain in what is essen- 
tially a start-up stage for 
several years* In such a 
case? evaluation might 
appropriately be limited to 
a process evaluation for the 
entire lifetime of the proj- 
ect? focusing on such issues 
as the most effective way of 
recruiting and training 
staff? or the best means of 
developing referral pools 
and initial intervention 
methods* Inferences about 
project effectiveness are 
only appropriate after some 
plausible interventions have 
been fully implemented* 

The_Tax3aier_gver_0ur 
Shoulders 

In this Alternative Edu- 
cation Program? the emphasis 
on evaluation has a basis 
that goes beyond its utility 
as a sound management prac- 
tice* The OJJOP has awarded 
grants to the alternative 
education action projects 
involved in SAES totalling 
I10?94^?^42* Congress and 
toe Office of Management and 
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Budget wtllingt some of 
these grants may be supple- 
mented in future years~even 
more public funds will be 
spent* In addition? OJJOP 
is spending over IBOOfOOO of 
taxpayers' money f or techn i - 
cal assistance and for eval- 
uation each year« This 
expenditure of publ ic funds 
is justified in large meas- 
ure on the basis of the 
knowledge that will be 
ga ined about al ter native 
education and delinquency 
prevent i on«. There is no 
justification for expending 
these funds without conduct- 
ing the most thorough and 
rigorous evaluation possi- 
ble* As the background 
paper for the OJJOP Al terna- 
t i ve Edu cat i on Program 
announcement (1930) put itf 
"Without standardized meas- 
ure? rigorous evaluation 
designs? and adequate fol- 
low-up time framesf we will 
continue to be unable to 
assess the effectiveness of 
alternative education for 
del inquency prevention* 
Policy and funding decisions 
will continue to be made 
without such knowledge" (p« 
^3)« Therefore? the Program 
Announcement required all 
appi icants to "provide assu- 
rances in their applica- 
tion(s) agreeing to cooper- 
ate with the national 



evaluators in terms of • • • 
the overall evaluation com- 
ponent" ( p« 9 ) • 

Scientific? practical? 
moral ? and programmatic con- 
siderations converge in 
demanding the most rigorous 
possible evaluation of the 
activities undertaken as 
part of the Alternative 
Education Program* If the 
projects and their umbrella 
organ i zat i on s vi ew the i r 
activities as demon strati on 
efforts in an area where 
knowledge of what works is 
desperatel y needed? they 
must consider evaluation an 
i ntegral and helpful aspect 
of project devel opment« 

To accomplish a rigorous 
evaluation? collabor ati on 
between project implementers 
and evaluators is essential* 
A structure designed to 
facilitate this collabora- 
tive process? Program Devel- 
opment Evaluation? is 
described in a subsequent 
chapter • The eval ua t i on 
process has required? and 
will continue to require? 
effort and resources on the 
part of each acti on proj ec t • 
This is not surprising; good 
evaluations are costly? time 
consuming? and demand i ng« 
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1. Throughout this chapter -treatment- » 

Shorthand description of any i ntervent.on intended to bring 
about an effect in a person, group, JV"^ -^s 

specifies the nature of the outcome intended, nor implies 
any particular modality of intervention. 

2. Each of the other possibilities 

explored at one time or another with at least one of the 
alternative education projects. 

3. Randomization may not be the ^^^^ "^^^''l^l^^^J^^ °^ its 
choice if a project is still in the stages of developing its 
intervention in potent form, if it is floundering in devel- 
ip^ng any intervention at all, or if the "^^^^^l^^^^",;!"- 
domization are such that the intervention itself is sub 
verted by experimentation. Tharp and Gallimore (n-^*) 
describe stages in project development -^^•'^^•"^f J^^. 
than true exoer i mentat ion may be most productive. But ran 
do^^zation ifgenerally the method ".-^^^.^^^Jf^" 
can implement some plausible intervention ^^^J^^. 
When it can continue to function if ^'^fsts. 
ent, and when doubt about the intervention's effects exists. 

^. This story came to our attention 4hen retold by Gott- 
fredson (1971). 
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Program Development Evaluation 



The Schodl Action Effec- 
tiveness Study (SAES) is 
faced with a tough? but not 
unusual challenge* Evalua- 
tions—not only delinquency 
prevent! on eval uat ions-- are 
commonly marked by weak 
interventions? or interven- 
tions of unknown strength 
and integrity; a lack of 
theory; a rapidly cha;iging 
project env i ronroent and 
changes in project goals? 
objectives? or methods; lit- 
tle commitment of project 
implementers to evaluation? 
and little understanding on 
the part of evaluators of 
the problems of implementa- 
tion; fears about the ways 
eval uat ion results may De 
used; a lack of sound meas- 
urement of the outcomes of 
interest; weak or nonexis- 
tent evaluation designs; 
ambiguities about goals? 
objectives? proolems? and 
needs; and inadequate 
resources (time and money) 
to cope effectively with all 
these pr obi ems • 

The history of previous 
delinquency prevention eval- 
uations? described by Ogawa 
(this volume)? is a history 
of evaluators and project 
implementers grappling with 



I am grateful for the advice 
of Deborah Daniels? Oenise 
Cm Gottfredson? and Jane St« 
John on a draft of this 
chapter? and for the discus- 
sions with J« Douglas Grant 
and Carol Yamasaki in the 
early stages of the creation 
of the Program Development 
Evaluation model* 



these problems* The success 
of SAtS will depend upon the 
extent to which it develops 
methods that avoid these 
d i f f i cul ties* The deve lop- 
men t of such methods has an 
importance that goes beyond 
the Alternative Education 
Program* E va 1 uator s and 
program devel opers every 
wnere need practical and 
sound methods for improving 
project impl e men tat i on and 
f OS teri ng more useful and 
r i gor ous evaluations* 

To meet its challenige? 
SAES must implement an eval- 
uation structure to meet the 
f ol 1 owing demands: 

1* Increase the likeli- 
hood that plausible and 
potent interventions will be 
i mpl emented* 

2* Make possible the 
assessment of the strength 
and fidelity with which 
interventions are imple- 
mented* 

3* Provide for the 
interpretation of experience 
i n theoret i cal terms* 

^* Document project 
plans and their implementa- 
tion as they evolve? recog- 
nizing that in actuality 
pi ans neither stay put nor 
are necessar i 1 y f ol 1 owed* 

5* Conduct the most rig- 
orous evaluation possible in 
terms of the strength and 
relevance of the design and 
the measurement of key out- 
come var i abl es* 
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6« Do all this with lim- 
ited resources in a short 
period of time* 

As i s typical of many 
evaluations? the evaluator 
is an "outsider*" That ist 
each action project competed 
independently for funding on 
the basis of the strength of 
its proposal • Evaluatabil- 
ity was not a major consid- 
eration in the funding deci- 
sions* Action projects are 
not under the control of the 
evaludtort nor did the eval- 
uator have any hand in the 
sel ection of action proj- 
ects# Under these clrcums- 
tancesf action project per- 
sonnel iiiay perceive 
evaluation as something 
imposed upon them by an 
alien and perhaps unfriendly 
dgent* Avoiding this per- 
ception is important? 
because we^expect that proj- 
ects will increase in effec- 
tiveness over time in pro- 
portion to their use of 
evidence provided by ti),^ 
eval uat ion« 

Instrumental in meeting 
these six demands? there- 
fore? is a further demand 
that SAES gain the coopera- 
tion of action project ' ^ 
impl ementers • This is 
essential to ( a ) increase 
the extent to which the 
evaluation is directed to 
the aims of each action 
project rather than focusing 
solely on the goals of the 
Federal sponsor? 
(b) increase the extent to 
which evaluation methods and 
results are used by the 
action project in its devel- 
opment? (c) ensure that 
action projects devote an 
appropriate level of 



resources and attention to 
evaluation? (d) increase the 
rigor and relevance of the 
evaluation? ( e) assist in 
clarifying goals? objec- 
tives? and plans to focus 
trie evaluation and to assist 
in project devel opment ? and 
(f) capture the rationale or 
theoretical perspective of 
each project? directing 
attention to the impl ica- 
tions of these perspectives 
for the development of 
delinquency prevention 
efforts* 

Implicit in this 1 ist of 
demands is the need for an 
effective evaluation to go 
beyond the two approaches 
common in many evaluations* 
Some evaluators approach 
trieir task in a wooden way? 
imposing a common set of 
measurement and design 
requirements that may be 
insensitive to the alms or 
circumstances of the action 
projects? and that are 
likely to be passively or 
even actively resisted by 
project i mpl ementers* Thi s 
approach seldom fosters 
project development* The 
evaluation may end up hope- 
lessly corrupted or may 
assess a set of interven- 
tions that were never imple- 
mented as intended* 

A second common approach 
is to conduct a flabby eval- 
uation* Unfortunatel y? a 
frequent response of evalua- 
tors to the six tough chal- 
lenges is to abandon rigor* 
Evaluations are often lim- 
ited to attempts to obtain 
flow data; efforts to Impl e- 
ment procedures leading to 
confident conclusions about 
effectiveness and to assist 




PDE 



in project development are 
abandoned* Weak evaluation 
designs often result in an 
inappropriate level of opti- 
mismt with little evidence 
of effectiveness after lots 
of taxpayers^ dollars have 
been spent* SAES aims to 
conduct the most rigorous 
possible evaluation while 
being flexible and useful to 
pro ject impl ementer s* 

The diverse demonstration 
projects involved in the 
Alternative Education Pro- 
gram aim to alter organiza- 
tional forms and educational 
experiences to prevent 
del i nquency* The^ommon 
goals and objectives of the 
programt specified by OJJOP 
(1980)f form a core or com- 
mon basis for the evaluation 
of all the projects in the 
national program* But proj* 
ects are run by community- 
based organizations? school 
systems? and a university? 
and each has distinctive 
problems and goals* Over- 
laidf therefore? on this 
common framework is a 
diverse set of organiza- 
tional envi ronmentSf goal st 
objectives? and interven- 
tions specific to each proj- 
ect* 

A Program Development 
Evaluation <POE) model pro- 
vides the structure for the 
evaluation of these proj- 
ects* The structure i s 
intended to anticipate and 
foster the development of 
these projects by involving 
project personnel in a cycle 
of evaluation activities* 
This structure is intended 
to (a) make rigorous evalua- 
tion possiblet (b) make the 
evaluation relevant not only 



to national concerns but 
also to the concerns of 
project personnel and manag- 
ers? (c) document project 
i mpl ementat i on? 
(d) facilitate project 
impl ementat i on? (e) tie the 
evaluation explicitly to 
del inquency prevention 
theory? and (f) integrate 
research with project opera- 
tions so that projects 
develop by using Xhe results 
of researc^i in project plan- 
ning* Rel a ted str uc tures? 
differing somewhat in empha- 
sis and detail? are provi ded 
by Empey (198 0) and Tharp 
and Gallimore (n*d*)* Those 
related structures are 
guided by some of the same 
concerns that led us to 
develop the PDE structure* 

The Program Development 
Evaluation model stems from 
the action research model* 
This approach assumes that 
the prospects for promoting 
change are greatest when the 
progr am dec i s ionmakers* 
stake in the research is 
made clear by their own par- 
ticipation in the research* 
Project dec i s ionmaker s and 
researchers col 1 aborate 
through a continuing dia- 
logue in which researchers 
provi de feedback on the 
consequences of project 
action* Action research 
involves a cycle of hypothe- 
sis formulation and plan- 
ning? action? evaluation and 
i nf ormat i on feedback ? and 
then renewed hypothes is f or- 
mulation and planning* As 
the cycle is repeated? and 
i nf ormat ion der i ved from 
project efforts and research 
is used in decision making? 
projects should become more 
effective — turning the pro- 
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cess into an upward spiral 
of activity. 

Projects usually change 
over time on the basis of 
the experience gained as 
they develop iWilkins £ 
Gottfredsont 1969) • What 
Pearl (1962) has called 
••quality control** is needed 
to insure not just that a 
program is run according to 
t he pi ant but that a plan 
exTsts and is modified to 
coincide with the way a 
project? as it develops? is 
actually run. Many attempts 
to demonstrate the effec- 
tiveness of specifiable 
social programs have failed 
in part because plausible 
interventions were not 
implemented or their imple- 
mentation was not documented 
(Uuayt 1977; Sechrestt 
Whitet t Browof 1979; Hall t 
LouckSf 1977 )f or toe plan 
for the innovation was not 
clearly articulated at the 
outset (Sarasont 197l)« 

The PDE model is espe- 
cially well-suited for 
facilitating and studying 
the development of a program 
by assisting in the planning 
process* It provides a 
mechanism by which an organ- 
ization can make its plans 
explicit? and then engage in 
self -study as it goes about 
impl ement ing them. It al so 
helps the evaluator monitor 
and document project plans 
and their implementation as 
the project evolves. In 
short? our Program Develop- 
ment Evaluation is an 
attempt to integrate evalua- 
tion and organ i national 
development* Its action 
research approach to know- 
ledge generation and organi- 
zational growth is derived 



from a tradition of concern 
for practical theory? useful 
research? and organizational 
change and development* 

Antecedent s^of_PDE 

One of the roots of Pro- 
gram Development Evaluation 
is the practice of organiza- 
tion development (0D)« 
French and Bell (1978) char- 
acterize 00 as a process 
invol ving act ion research 
that emphasizes normative 
change? is based in beha- 
vioral sc ience? i nvol ves 
experience-based learning of 
intact work teams? and 
emphasizes goals and objec- 
tives* By characterizing 00 
as a process? French and 
Bell mean that 00 is "not to 
be regarded as a one-shot 
solution to organizational 
probl ems ? but more as a 
•growing toward* greater 
effectiveness through a 
series of intervention 
activities over a period of 
time. • • • Changing the 
culture of • • • an entire 
organization is a long-term? 
involved process (p« 69)«" 
In addition? they see 00 as 
a process involving 
rational? empirical strate- 
gies? but one that is even 
more dependent on normative- 
reeducative strategies: 
"The c1J.ent defines what 
changes and improvements he 
or she wants to make? rather 
than the change agent; the 
change agent attempts to 
intervene in a mutual? col- 
laborative way with the 
client as they together 
define problems and seek 
solutions; anything hinder- 
ing effective problem 
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solving is Drought to light 
and publicly examined (pp* 
75-76) The emphasis on 
normative education is based 
on the assumption that 
behaviors are rooted in 
normsf values? or beliefs as 
well as in rationality and 
self-interest* OD i s a 
data-based approach to 
planned change in which 
information is a spur to 
action* Unpleasant informa- 
tion is not to be avoided 
but rather treasured oecause 
it may lead to advancement? 
to clarification of prob- 
lems* Typically? 00 empha- 
sizes concrete goal setting 
through the shared experi- 
ence of a group in formulat- 
ing plans* The on-the-job 
learning experience of an 
intact group is presumed to 
promote organizational and 
individual effectiveness* 

The interactive? collab- 
orative* part i c i pat i ve 
approach* often used by 
behavioral scientists or 00 
specialists serving as con- 
sultants or facilitators of 
organizational planning and 
decision making? has much to 
offer in overcoming some of 
the difficulties an evalua- 
tion may expect to face* 
Firstf increasing an organi- 
zat i o'^* s ef 1 ec t i veness 
should increase the likeli- 
hood that it will succeed in 
i mpl ement i ng in t erven t ions 
with a possibility of being 
shown to be effective when 
subjected to serious summa- 
tive evaluation* Second? in 
the 00 process? the evalua- 
tor approaches an organiza- 
tion in a manner that may 
decrease the extent to which 
he or she is perceived as an 
alien invader* 3y helping 



an organization clarify its 
goals and objectives? by 
assisting in creating open 
communication about prob- 
lems? and by fostering the 
expectation that projects 
will change and develop over 
time? tne eyaluator may come 
t3 oe consi Jr^red more as an 
insider? an tity to be 
trusted to ( ey useful 
news* And? perspective 
that informa ^n? even 
uncomf or tabl e i nf or mat i on? 
is valuable in fostering 
growth and confronting 
important problems may 
decrease the organization's 
usual fear of evaluation* 
Finally? the links between 
00 and action research make 
the interjection of formal 
evaluation possible* 

The Program Development 
Evaluation model is in part 
a descendent of an OD Tiethod 
previously used by the 
Social Action Research Cen- 
ter (Blanton L Alley? 1975) 
in a series of projects to 
manage and study social 
change* This predecessor? 
called the Program Develop- 
ment (PO) model? Mas devel- 
oped through attempts to 
evaluate human service proj- 
ects* In this model? feed- 
back is a mechanism of proj- 
ect development that 
i nvol ves moni tor i ng a proj- 
ect's environment? the 
implementation of strate- 
gies? and the achievement of 
goals* In practicet the 
Program Development special- 
ist focuses on interaction 
with project implementers to 
assist in assessing needs? 
in articulating goals and 
more specific objectives? in 
analyzing a project's force- 
field (environmental con- 
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straints and resources)f and 
in developing strategies for 
change or implementation. 
Blanton and Alley (1975f 
Chap. 7) distinguish three 
kinds of evaluation possible 
using PO concepts: 

(a) evaluation of relevance? 

(b) procedural evaluation? 
and (c) outcome evaluation. 
Although they discuss poten- 
tial structures for outcome 
eval uationv emphas is has 
been placed primarily '^n 
other aspects of the organi- 
zation development process. 
The 90 model is illustrated 
in Figure 1. In applica- 
tionf great emphasis has been 
put on the participatory 
nature of this process and 
on avoiding intrusive moni* 
toring procedures to enhance 
the credibility of evalua- 
tion designs. Participation 
and unobtrus iveness facili- 
tate the implementation of 
the planning portions of the 
PD process? and reduce the 
possibility that PO will 
become an unwelcome or bur- 
densome appendage. Like 
other forms of OOf PD empha- 
sizes participatory planning 
in part to foster normative- 
reeducation and in part to 
increase organizati onal and 

i ndi vidual competenc i es in 
decision making and plan- 
ning. 

Both 00 and the present 
eval uation have roots in 
action research. According 
to French and BeVi (1978)f 
the origins of action 
research lie in the work of 
Oewey ( 1933) ♦ Col 1 ier 
(I945)f and Lewin (1946). 
The roots of action 
research? however f are 
deeper than this. They can 



be traced back to the Baco- 
nian formulation of the 
scientific method? which 
specified three steps: 
(a) the formation of 
hypotheses? (b) the empiri- 
cal testing of the 
hypotheses? and (c) the 
acceptance or rejection of 
the hypotheses (Deese? 
1972). Action is taken to 
"twist the lion^s tail** to 
learn aoout nature. Since 
Bacon? science has been 
active rather than specula- 
tive? hi stor ical ? or ref 1 ec- 
tive. Oewey translated the 
scientific method of problem 
solving for laypersons? and 
Collier and Lewin both 
applied the scientific 
method to solving practical 
social problems. 

Collier? a commissioner 
of Indian Affairs concerned 
with improving race rela- 
tions? wrote of action 
research? cla iming that: 

Research and then more 
research is essential to 
the program? that in the 
ethnic field research 
can be made a tool of 
action essential to all 
other tools? indeed that 
it ought to be the mas- 
ter tool. But Me had in 
mind a particular kind 
of research? or? if you 
will? particular condi- 
tions. We had in mind 
research impelled from 
central areas of needed 
action. And since 
action is by nature not 
only specialized but 
also integrative. . 
our needed research must 
be of the integrative 
sort. Again? since the 
findings of the research 
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must be criticized by 
them through their 
experience* the adminis- 
trator and the layman 
must themselves partici* 
pate creati vel y in the 
research? impelled as it 
is from thei r own area 
of heed* 

(Collierf I945f cited by 
French L Bel If 1978? p. 

9^ ) , 

Broader attention was 
called to action research by 
Lewinf an eminent and 
inf I uent iai psychol ogical 
theorist with a keen inter- 
est in the applications of 
psychology. He saw that 
cooperation between the 
change agent (or field 
worker) and the researcher 
is important for both plan- 
ning and management: 

Planning starts usually 
with something like a 
genieral idea* For one 
reason or another it 
seems desirable to reach 
a certain objective* • • 

• The first stepf then» 
is to exami ne the i dea 
carefully in the light 
of the means availaDle* 
Frequenti y more f act- 
find i ng about the situa- 
tion is requ ired. If 
the first period of 
planning is successful 
two items emerge: an 
•overall pi an* of how to 
reach the objective and 
a decision in regard to 
the first step of the 
action* Usually this 

pi anning has al so somew- 
hat modi f ied the or igi- 
nal idea. The next 
per tod is devoted to 
executing the first step 
of the overall plan 

• • • (and) by certain 
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fact-findings* • • • 
This • • • fact-finding 
has four functions* It 
should evaluate the 
action by showing 
whether what has been 
ach i eved i s above or 
below expectation* It 
should serve as a basis 
for correctly planning 
the next stept {for} 
modifying the •overall 
plan** Finally? it gives 
the planners a chance to 
1 earn? that is? to 
gather new general 
insight • • * regarding 
the strength or weakness 
of certain * * • tech- 
n i ques of action* • * * 

Rational soci al man- 
agementf thereforet pro- 
ceeds in a spiral of 
steps each of which is 
composed of a circle of 
pi anningv act ion f and 
fact-finding about the 
result of the action* 

(Lewinf I9^7f pp 
333-334). 

Th i s sequent i al and sp i - 
raling model of problem 
solving is now widely used 
i n organi zat i orial devel op- 
ment effortSf and has been 
applied in a variety of 
i ndustr i al f human servicet 
and educat i onal ac t i on 
research projects; and it 
appears to be at the heart 
of Tharp and Gal 1 i mor e* s 
( n*d* ) Eval jation Succes- 
sion model* 

Several varieties of 
activity are often called 
act i on research ( Che i nt 
Cookf C Hardingf 1948)* 
Sometimes the effort is lim- 
ited to di agnos is and recom^ 
mendations; someti mes 
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organizations or project 
iwplementer s carry out the 
entire process; sometimes 
records or diaries of 
actions taken and their per- 
ceived effects are main-* 
talned. As Tharp and Galli- 
more (n.d.) note? there are 
several ways of "knowingf" 
each appropriate to differ- 
ent stages in the develop- 
ment of a program* What 
they call "experimentationf 
••qual i tati ve/personal know- 
ingt" "data guidance?** and 
••program evaluation" are all 
useful in prggram develop- 
ment and evaluation. But 
the variety of action 
research most productive of 
trustworthy knowledge i s 
exper i mental act ipn 
research. Unf ortunatel y f 
experimental action research 
is also tha most difficult 
to perform? because it 
requires the conditions 
necessary for confident 
inference (see the preceding 
chapter)? and a stable set 
of interventions that the 
organization knows how to 
and can implement in testa- 
ble form. Seldom do 00 
efforts aim to i mpl ement 
ex per i menta 1 ac t i on 
research? largely because it 
is so difficult. Implement- 
ing exper i Tiental act ion 
research is? however? a 
chief aim of the School 
/^xtion Effectiveness- Study. 
Eval uat ion must be coordi - 
nate rather than subordinate 
to problem solving; solving 
probl ems wi thout 1 earn ing 
how or why they were solved 
will not accompi i sh the aims 
of the Alternative Education 
Program or of SAES* 



To conduct the School 
Action Effectiveness Study? 
we have built on the Program 
Development model? but have 
altered it in major ways to 
make this ID tool m*ore 
appropriate for an evalua- 
tion. In particular? the 
POE model emphasizes to a 
far greater extent 
(a) theor y? ( b) measurement? 
and (c) experimental or qua- 
si -exper i mental des i gn. In 
addition? some terms (most 
notably ••ob ject i ves*« ) have 
been redefined? and a struc- 
ture for documenting project 
implementation has been 
added^ At the same time? 
the new structure retains 
the act i on research emphas i s 
on a cycle of development 
activity that was central to 
P3. 

The resulting Program 
Development Eval uat ion 
model? illustrated in Figure 
2? thus incorporates theory 
as an exolicit component? 
gives jneasureable goals and 
Ob jectTve~a"more hard-nosed 
meaning* incorporates plan- 
ning for evaluation imple- 
mentation in the same way 
that planning for any other 
aspect of a project is 
i ncorpor ated? and al I ows 
project implementers and 
evaluators to monitor criti- 
cal benchmarks in the imple- 
mentation of any strategy to 
create change. The princi- 
oal concepts involved in the 
POE structure are listed in 
Table It and each is elabo- 
rated below. 
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PDE Conc epts 

The Alternative Education 
Program involves a common 
set of goals; directed pri- 
marily at the problems of 
delinquencyt dropout? and 
nonattendance in school • 
But a fundamental tenet of 
the action research paradigm 
is that the implementers of 
an individual project should 
be actively involved in 
creating the research proj- 
ect and setting its goals* 
^Farthermoref seldom do the 
aims of any particular 
action project overlap fully 
with the aims of the sponsor 
of this program* Thereforet 
interaction with each proj- 
ect must begin with an 
exploration of its intent. 

l-£25ll£!5i. an^- Go als • Most 
organizationsf and most 
projects? have multiple 
aims. Within the POE frame- 
Morkf a general or overarch- 
ing aim is called a goal. A 
goal is the obverse of a 
problem; it specifies how 
the level of the problem may 
be measured and therefore 
how one may know if progress 
is being made* Several sec- 
ondary questions are impor- 
tant when discussing goals. 
The first question— how each 
goal may be measured— serves 
to reduce ambiguity and ena- 
ble evaluation. The second 
question serves to promote 
realistic planning; it asks 
when a project can realisti- 
cally expect to make a sub- 
stantial difference. And 
the third question? essen- 
tial i n expf er imental or qua- 
s i-exper i mental act i on 
researchf asks how one may 
know that the project itself 
was responsible for progress 



towards the goal . These 
quest ions are f of course? 
steps toward involving proj- 
ect implementers in the 
design of the evaluation. 

Theory. Actions are 
taken for reasons that are 
either articulated or unar- 
ticulated. The PL)E struc- 
ture is a vehicle for making 
theory explicit. This is 
useful because* as the Panel 
on Research on Rehabilita- 
t i ve Techni ques (Marti n? 
Sechrestf C Redn^^r t 1981 ) 
notes: 

In attempting to solve 
any problem? a clear 
idea of the nature of 
the probi emf its causes f 
and devel op mental pro- 
cesses is vital. In the 
absence of an adequate 
conceptual framework . « 
. the rush of enthusiasm 
for an interestincj 
intervention is likely 
to short-ci rcui t cons i d- 
eration of these fac- 
tors. The result is . . 
. ef f or ts that may be 
unrelated to the causes 
of crimef ignore the 
most suitable target 
populationSf and fail to 
consider questions^ of 
optimal tining and 
strength of the inter- 
vent i on . The adopt i on 
of a theoretical frame- 
work necessar i 1 / prompts 
considerat i on of the 
above factors andf one 
hopeSf thoughtful devel - 
opment and impi ementa- 
tion of . . . intervene 
t ionSf thereby 
i ncreas i ng the chances 
for effectiveness 
(P- 29), 
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Theory helps to organize 
knowledge and to cofnmuni- 
cate? it provides a quide 
for actiont and it assists 
in developing and assessing 
interventions. ••Once a 
basic proDlem is stated in 
theoretical terms? planners 
h3ve an explicit foundation 
on which to build an inter- 
vention strategy and from 
which to derive a research 
strategy i n conjunct i on wi th 
the intervention** (Martin et 
al.f I98lf p. 3^; cf. 
Glaser* I9e0)« In shortt an 
explicit theory provides a 
template for project imp1e- 
menters* use in building 
their interventions? as well 
as a template by which both 
impl ementers and researchers 
can assess those interven- 
tions« Therefore^ the POE 
process calls for deliberate 
and careful consideration of 
the question? ••Why do these 
problems cixist?" 

Ob ject i yes« In the lan- 
guage of POEf an objective 
is an i n termed i ar y outcoiie 
that a project's theory of 
action implies is impor- 
tant.<l> Like goalSf objec- 
tives must be stated in mea- 
surabl e terms. 

Some examples may hel o 
make the distinction between 
goals and objectives clear. 
Suppose that^ a change agent 
wishes to decrease the death 
rate due to gastroenteritis 
in a rural society. The 
change agent theoriz'^s that 
the suffering and death are 
due to the contamination of 
village water supplies with 
the cholera mi cro-or gani Sin . 
This theory might suggest a 
campaign to chlorinate 
wellSf with the objective of 



decreasi ng th i s 
contaminati on. The ohjec- 
tive would be neasjred by 
1 aboratory anal yses of 
w el 1 -water samo 1 es to det er- 
mine the levels of microbial 
contamination? and the godl 
might be measured by counts 
of deaths per 100*000 popu- 
lation due to gastroenteri- 
tis. Another change agent 
might see the problem somew- 
hat differently. This sec- 
ond change agent may theor- 
ize that the suffering and 
death are due to poor envi- 
ronmental san i tat i on: 
Because few v i 1 1 agers use 
sanitary la tr in est we 11 
water is easily contaminated 
and th'3 c hoi era mi cro-organ- 
ism spreads from infected to 
un i nf ected persons. Thi s 
theory might suggest an 
env i ronment a1 san i tat i on 
campaign directed at per- 
suading vi 1 1 agers to con- 
struct sanitary latrines and 
sanitary wells. The objec- 
tive now involves v i 1 1 aijer 
behaviorf and might be meas- 
ured by the proportion of 
households using sanitary 
latrines and water from pro- 
tected wells. A theory cant 
of coursef suggest niultiale 
interventions i^nd multiple 
objectives. The second 
change agent's theory would 
also reasonably imply chlo- 
rination of wells and 
assessments of well water* 
The more comprehens i ve -i 
theoryf the more complex the 
array of i n tervent i ons and 
ODjectives it is likely to 
s jggest. 

Change agents could 
develop theories at many 
levels to explain the prob- 
lem of cholera deathSf and 
each level would suggest 
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somev^hdt different 
i nterventions« To continue 
the examplest change agents 
might attribute the problem 
to (a) normative beliefs in 
village societies that cur- 
rent standards of environ- 
mental sanitation are ade- 
quate? (b) the poverty and 
segregation of the rural 
people? which deprive them 
of the resources to build 
sanitary devices and concen- 
trates them so that they are 
at high risk? (c) social 
stratification that allows 
only an elite merchant class 
access to sufficient 
resources to enjoy a sani- 
tary environment? 
(d) stratification in the 
world system that enables 
capitalist countries to keep 
countries with rural rubber- 
tapping populations impover- 
ished and the cost of raw 
materials low. Each of 
these theories may have con- 
siderable validity* Yet 
each would imply different 
interventions to solve the 
problem? ranging from du.iip- 
ing chlorine in wells to 
overthrowing the capitalist 
world system* No single 
cholera prevention project 
is likely to attempt inter- 
ventions at all of these 
levelst and so will not have 
objectives at each level* A 
project 's theory of 
action — tie theory that 
drives its interventions — is 
the theory that is relevant 
in specifying objectives. 

Again? answers to ques- 
tions--how objectives may be 
measured? when effects are 
to be expected? and how one 
may know that the i nterven- 
tion caused the effects — 
-serve to create the evalua- 
tion design. 



lQiSL!£§Q£i2Q« » nter- 

vention is an action taken 
to achieve an objective or 
set of objectives* Ordinar- 
ily? it is a major component 
of a pro ject* The term is 
often s ynonymous with 
••change? ••treatment ?•• or 
"componen t • Some interven- 
tions are aimed at changing 
the behavior? attitudes? or 
status of individual peopl e ; 
others are ai med at changing 
tfie behavior of an organ i za- 
tion or collectivity*<2> An 
5 *4tervent ion is a process? 
action? structure? rulet or 
substance that a project 
applies or puts in place to 
achieve an objective or set 
of objectives? and therefore 
to move closer to achieving 
its goal(s). An interven- 
tion may be chemicals physi- 
cal? biol og i cdl ? behav i or al ? 
social? politicalf or struc- 
tural * 

Forcef i el d * A forcefield 
is the soci al -psychol ogical 
field that immediately sur- 
rounds a decision or action* 
It includes the forces that 
compel or restrain against 
alternative actions? as they 
are perceived by an indivi- 
dual or c or porate actor * 
The not ion of a forcefield 
CDmes from Lewin's (1951) 
i deas about the field of 
forces influencing action. 
An examination or analysis 
of an organization's force- 
field? especially one that 
focuses on the f iel d in 
terms of the resources 
avail able and the obstacles 
to ac t i on ? is f requenti y 
useful for four reasons: 
(a) By focusing on the 
organi zat i on' s perceptions 
of environmental i nf 1 uences ? 
the nature of these percep- 
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tions becomes explicit and 
open to scrutiny? revisiont 
ammendat iont suppi ementa- 
tidof and test. (b) A com- 
plete account of obstacles 
and resources decreases the 
likelihood that either pit- 
falls or potentials will be 
overlooked in the develop- 
ment of a project. 
(c> Using knowledge of the 
influences in the project^s 
environment helps to capi- 
talize on opportunities or 
arrangements that go beyond 
the resources under a proj- 
ect's direct control, 
(d) Alternative strategies 
-or plans to implement any 
intervention can be created 
and assessed in the context 
of the f orcef iel d.<3> 

Practical gui dance y on 
working with an organization 
tp analyze its forcefield is 
provided by Blanton t Alley 
(I975f 103-113). 

because i n i t i al anal yses 
of a forcefield may be 
Ob jec t i vel y incorrect t 
because perceptions change 
over timet and because the 
action of a project may 
alter its forcefield? the 
dynamic nature of the field 
is to be expected. A sensi- 
ble practice? therfore? is 
to renew forcefield analysis 
per iod ical 1 yt espec i al 1 y 
when any strategy being exe- 
cuted on the basis of an 
initial forcef iel d anal ysi s 
i s not work i ng wel 1 . 

iil^tSSLiSi* Strategy es 
are plans.<^> According to 
the PDE model f strategies 
are developed from a force- 
field analysis^ just as 
objectives and interventions 
derive from a theory of 
action about a problem. 



Several possible strategies 
for implementing a project 
or one of its component 
interventions are likely to 
exist. The task for project 
iinplementers and those who 
are attempting to facilitate 
strategy development is to 
create a plan that is per- 
ceived as feasible and 
attractive. If a critical 
path in sowe plan is blocked 
and no way around the obsta- 
cle is perceived? the plan 
is not a good one. Alterna-- 
tive paths that objectively 
exist but have not been per- 
ceived will not be followed. 
(This point illustrates why 
thorougn and creative force- 
field analysis is helpful*) 
A strategy that appears 
workable will make use of an 
organization's resources to 
overcome the obstacl es to 
i Tipl ementat ion. Such a 
strategy may involve 
(a) moving around an obsta- 
cle? (b) decreasing the 
strength of the forces work- 
ing against implementation? 
(c) turning an obstacle into 
a resource? or (d) involve a 
strategy in which the obsta- 
cle is irrelevant and need 
not be overcome. 

A fully articulated stra- 
tegy is composed of two 
kinds of elements: critical 
benchmarks and tasks. 

Cr rticaL_benchs!jarks. A 
cr itlcal "benchmark is a key 
dec is ion? agreement? act i on? 
or arrangement necessary to 
move forward with a plan. A 
benchmark is much like a 
gate that must be opened to 
nnove al ong a path.<ii> If the 
gate does not open* progress 
in executing the strategy is 
blocked. The locations of 
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these benchmarks (or the 
nature of them) are made 
clear in the process of ana- 
lyzing the forcefield around 
an intervention* For exam- 
plef the forcefield analysis 
about a project's efforts to 
provide in-service training 
for teachers might impi y 
that an obstacle lies in 
teacher unwillingness to 
participate in training out- 
side of normal working 
hours 9 and that a resource 
is t!-£i authority of the 
deputy superintendent of 
schools to grant release 
time and to allocate the 
funds for substitute 
instructors* The deputy 
superintendent's agreement 
to grant release time and to 
authorize the expenditure 
for substitute teachers 
would then become a critical 
benchmark* The deputy 
superintendent is a gate- 
keeper CLewint__l947f p* 333) 
whose psychology must be 
examined to learn how to get 
the gate opened* 

Specifying when a criti- 
cal benchnnairk is to be 
accompi i shed provides a man- 
agement t3ol* Any strategy 
will require d temporal or 
logical sequence of mile- 
stones thdt must be met* In 
this examples a failure to 
accomplish this critical 
benchmark would signal the 
need to devise a new stra-* 
tegy for getting the train- 
t ng done? or the need to 
seek an alternative to 
traini ng* 

Tasks* The second pdvt 
of a strategy is the collec- 
tion of tasks required to 
execute it* A task sta.te- 
ment specificies who will do 



what by atlSQ*^*^ Specifying 
a person to be responsible 
for executing a particular 
taskf even when a group will 
be involved? promotes clar- 
ity* And specifying when a 
task is expected to be com- 
pleted is an additional man-* 
agement tool * 

Cr i tical benchmarks and 
tasks both serve important 
, f unct i ons in project manage- 
ment and worker reinforce- 
ment: They serve to guide 
an organ i zati on 's efforts • 
They provide one kind of 
objective standard of 
achievement* A 1 ack of such 
objective standards 
••deprives the workers * * • 
of their legitimate desire 
far sati s fact ion on a real - 
i St ic bas i s« Under these 
c i rcumstances f sat i s f act i on 
or di ssat i sf act i on wi th 
(one' s) own achievement 
becomes mainly a question of 
temperament (Lewint I9^6f p* 
35) *" 

2£2££L2B!!!SQi* very 
heart of""the""pOE model is 
the expectation that project 
development will be an ongo- 
ing processf and that the 
pro ject' s env ir onment i s 
dynamic* Only an effete 
organization is immobile? at 
equil ibrium* Tension? 
reassessment? review? 
replanning? and changes in 
actions taken are the hall- 
marks of vigorous projects* 
Con sequent 1 y ? PDE is a 
cycl ical process of action 
research as progress is made 
towards ach i evi ng goal s and 
objectives (or as goals and 
objectives are redefined)? 
as new information becomes 
available? and as the envi- 
ronment changes* 
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Oevelopment occurs 
largely through the use of 
informat ion* Information 
about the achievement or 
nonachievement of critical 
benchmarks signals that the 
forcefield has been usefully 
understood? or that develop- 
mental effort is required to 
reassess the organization's 
f orcef ield» Inf ormati on 
that an objective is being 
achieved signals that an 
intervention is effective? 
and i nf orwat ion that an 
objective is not being 
achieved signals a reconsid- 
eration of the appropriata- 
ness? strength? or f idel ity 
of the intervention^ and 
prompts new planning* 
Information that there is 
progress towards a goal sig- 
nals that the organization 
is on the right track* 
Information that there is no 
progress towards the goal 
may si gnal several things? 
depending on the pattern of 
other feedback* If inter- 
ventions are being imple- 
mented as intended and they 
are achieving their objec- 
tives? the theory is called 
into question* If objec- 
tives are not being met? 
either the theory or integ- 
rity of the intervention? or 
both? should be scrutinized. 
Success in bringing about 
elusive objectives and solv- 
ing serious problems is not 
to be expected at once* But 
the PDE structure is 
intended to provide inter im 
feedback on progress to ena- 
ble a strengthening of the 
project* 

£ val uat i on* The POE 
structure i s i n tended to 
facilitate several kinds of 
evaluation* The explication 



of a theory of action allows 
an assessment of its plausi- 
bility? and an assessment of 
the plausibility or strength 
of the project's planned 
interventions in light of 
the theory* 

9y tracking the achieve- 
ment of critical benchmarks? 
the structure allows assess- 
ment of the integrity with 
which an intervention is 
executed — it provi des eval u- 
ators and project managers 
with feedback on progress 
t3wards executing strate- 
gies* These are key ele- 
ments of formative evalua- 
t i n* 

The POe structure is also 
intended to facilitate rig- 
orous summative evalua- 
tion — it is experimental 
action research (Chein? 
Cook? e Harding? 1948) or at 
least quasi-experimental 
action research* It repeat- 
edly asks the question? "How 
do you know your interven- 
tion (project) made the dif- 
ference?*' The impl ementa- 
tion of an evaluation design 
is treated in the same way 
as the implementation of any 
other intervention* Essen- 
t iall /? the POE model 
assumes that evaluation is 
an essential component of 
effective project develop- 
ment and should receive 
coordinate effort with other 
aspects of project implemen- 
tation* Therefore? force- 
field analysis is oerformed 
for design and data collec- 
tion issues just as it is 
for any other project compo- 
nent* Because project 
Implementers are involved in 
the research design and in 
the specification of the 



-61- 



POE 

research questions* their 
comni tment to strong eval ua- 
tion is expected to 
increase* And? because the 
forcefield anal /sis focuses 
on the project impl ewertter s* 
own forcef i el d--- thei r per- 
ceptions of the possi- 
ble—the immediate environ- 
ment of the evaluation is 
taken into account *#hen the 
evaluation is desi'^nedf per- 
haps mitigating some of the 
resi stance to eval uati on 
act i vi t i es commonl y encoun- 
tered among impl ementers* 

L i mi t atjons and Po tent i al 
Cri t ic ji.sms 

The POE structure in its 
current state of development 
has some limitationsf 
creates some tensions? and 
is open to criticism. The 
most important appear to be 
that it is complexf it is 
time and expertise inten- 
sive* it does not yet 
directly assess the strength 
and fidelity of interven- 
tions* it fails to com- 
pletely resolve the tensions 
summative evaluation causes 
for project impl ementersf it 
is an imperfect mechanism 
for coping with the separate 
goals of project sponsors 
and implementing organiza- 
tions when these are not 
completely in accordf and it 
confronts researchers and 
implementers with tough 
decisions involving the 
sacrifice of rigorous 
research designs in order to 
achieve some aspect of proj- 
ect impl ementat ion« 



The human behavior 
requi red to successful 1 y 
i mp 1 ement the POE model i s 
complex* and the model •s 
i mpl ement at ion cal 1 s for a 
large investment of human 
resources* Use of the PDE 
structure cal Is for high 
levels of interpersonal com- 
petency? tact f pat i encef 
communication sktl 1 f and 
under standi ng of group rel a- 
tions in organizations* In 
addition? it calls for a 
thorough understanding of 
evaluation me thods — measure- 
ment* social science theory? 
exper imental and quas i-ex- 
perimental methods? statis- 
tics? and rigid adherence to 
schedules* Ironical ly? this 
combination of competencies 
are rarely found in one and 
the same person? suggesting 
that a team of workers may 
be required to conduct 
action evaluation using PDE* 
Furthermore? the cyclical or 
developmental natures of PDE 
requi res constant (or at 
1 east frequent per i odic ) 
attention* moni tor ing? 
updating? and i nf or ma tion 
communication* Thi s inten- 
sive and taxing arocess goes 
beyond the effort typically 
expended in an evaluation* 

Some trade-offs are 
likely tp be involved in 
staffing an evaluation using 
the POE itodel * Finding 
staff members with the 
requisite skills can be dif- 
ficult* implying that train- 
ing will be required* Oar 
experience impl ies that 
para-prof ess ional s can f unc- 
tion effectively using the 
model? but that they will 
require assistance with the 
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more technical evaluation 
and statistical issues. It 
also implies that social 
scientists trained primarily 
in research methodology f 
statisticsf and theory can 
successfully implement the 
model f but that they require 
a different kind of addi- 
tional training to do so. 
By using training to develop 
competencies f and a staff 
composed of persons with 
diverse ski 1 1 s t personnel 
costs may be kept relatively 
low despite the scarcity of 
persons with all the compe- 
tencies required* 

Sj^renath_and_Fidelit^ 

The POE model makes pos- 
sible the assessment of 
strength and fidelity 
through judgments about the- 
oretical plausibility and 
benchmark monitoring* This 
assessment can occur in two 
ways* Firstf project imple- 
menters can assess the con- 
sistency of their interven- 
tions and objectives with 
the theory of action under- 
lying their project* That 
iSf a project implementer 
can determine whether the 
objectives sought accord 
with the theory? and whether 
the interventions planned 
will plausibly achieve tha 
project • s ob jec t i ves* In 
shortf theory is a template 
for making judgments about 
the appropriateness of 
interventions and objectives 
that project implementers 
can use to quality control 
their own projects* Secondf 
observers of a project can 
assess its a pr io rj strength 
by determining whether the 
planned interventions will 
plausibly lead to the objec- 



tives or goals of the 
project by assessing them in 
comparison to state-of-the 
art theories in the field 
and the history of similar 
projects that have been con- 
ducted in the past* 

In implementing ?0E in 
the pastt we have not» how- 
ever? typically attempted to 
observe directly the admin- . 
istration of treatments or 
the conduct of interven- 
tions* As the utility of 
••manipulation checks'* in 
exper i mental soci al psychol - 
ogy implies* the direct 
assessment of treatments 
would be desirable* The 
Tharp and Gallimore (n*d*) 
account of evaluation suc- 
cession implies that moni- 
toring the integrity of 
iiterventions is more impor- 
tant in some stages of the 
development of a project 
than in others* At some 
points in a project's devel- 
opmentf evaluation issues 
will have to do with ideas 
for interventions or with 
strategies for getting an 
i nnovat i on adopted* At 
other pointsf evaluation 
issues will have to do with 
the integrity of the inter- 
vention's implementation and 
with the assessment of 
effectiveness * 

Accordinglyt we plan to 
pay more attention to the 
documentation of interven- 
tions as implemented where 
appropriate in the future* 
This documentation may take 
tne form of detailed manuals 
for the administration of 
treatments or programs; 
descriptions of the charac- 
teristics of staff and tar- 
get groups; and accounts of 
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the duration and scheduling 
of treatments or events? 
treatment protocols? or pro- 
portion of the population 
served^ In many cases 
direct observation or mini- 
ethnographies may be help- 
ful • 

Some shift in emphasis in 
the appi ication of the^ POE 
model is desirablet because 
in the first year of this 
evaluation we have focused 
primarily on the steps lead- 
ing to the adoption of a 
practice or innovation 
rather on the steps 1 eading 
to the integrity the 
innovation once ""adopted*** 
One minor shift in emphasis 
to increase the ability of 
the POE model to assess 
integrity is to give more 
emphasis to those aspects of 
project planning that are 
aimed at achieving integrity 
of an intervention* For 
example? POE plans could 
easily include strategies to 
develop manuals to guide 
service delivery? the making 
of diagrloses? and the train- 
ing of staff* Similarly? 
POE plans could include 
strategies to monitor staff 
performance? provide incen- 
tive structures to keep per- 
formance according to speci- 
fications? and the like* 
The implementation of those 
strategies would likely have 
two consequences: (a) the 
plans and their execution 
would increase the integrity 
of the inter vent ion? and 
<b) the information gener- 
ated by the implementation 
of these plans would 
descr i b e the integrity of 
interventions* 
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Tens i on appear s endemi c 
i n summat i ve eval uati on 
efforts* Too often in the 
past? evaluation has been 
used as a tool for cancel 
ling a project--even when 
pos i t ive eval uati ons coul d 
not reasonably be expected 
at an early stage of project 
development* Tension is 
also created by the inhe- 
rently political environment 
of action projects? and by 
environments where the suc- 
cessful project does not 
rock the boat* Rigorous 
evaluation requires the 
expenditure of time and 
m3ney? and often implies the 
necessity of arrangements 
that are di sr uptt ve* 

□ivergent^Goals 

Although the PDF struc- 
ture is explicitly designed 
to focus on the goals and 
objectives of an action 
project? at present the 
goals and objectives of the 
sponsoring organization must 
be overlaid on these proj- 
ect-spec i f ic aims- -and the 
overl ap is somet imes imper- 
fect* This is not a problem 
for an organization conduct- 
ing i ts own devel opment 
effort? but it creates 
resentment or resistance in 
some cases when a sponsor 
needs evaluation information 
that goes beyond what the 
implementing organ i z at ton 
sees as relevant* 

Tradeof f s_and_Research_Ri,gor 

Program development eval- 
uation is value laden* Par- 
ticipation of project imple- 
menters i s a fundamental 
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principle in the POE 
process; pursuit of the 
qoals and objectives of the 
implementing organization 
are generally assumed to be 
desirable (although open to 
question) • Furthermoref an 
aim of POe i s to develop the 
implementing organization's 
capacity to accomplish its 
aims* Therefore? evaluatars 
and implementers collaborate 
in evaluation design? ques- 
tion formulationf and plan*- 
ning« As a result? evalua- 
tors extensively intervene 
in project development—in- 
deed they become a part of 
the project* 

Some evaluators (Perlofff 
1979) see this as undesira- 
ble in a summative evalua- 
tion because it raises ques- 
tions about the 
generalizabi 1 i ty of the 
results to situations where 
evaluators are absent* In 
addition? just as evaluation 
needs sometimes intrude in 
project operations by creat- 
ing new tasks or striictufal 
arrangements? the pursuit of 
a project's programmatic 
activities very often 
results in compromises in 
research design* As Deutsch 
(1968f p* ^66) saysf ••The 
danger that confronts the 
research worker in such 
situations is the possibil- 
ity that his research design 
or methodology will be 
sacrificed to the achieve- 
ment of the social-action 
ob ject i ve*" 

This ••danger* may account 
in part for the reluctance 
academic soc i al sc ient i sts 
have shown to participate in 



action research* This dan- 
ger seems a small price to 
oay in exchange for the 
opportunity action research 
creates to contrit>ute to the 
solution of social problemst 
although the tradeoffs 
involving evaluation rigor 
are painful* In short* the 
POe model is no panacea for 
this tough problem. 

Compl^exitY 

The POE model is complex* 
A comparison of this model 
with the PO model (Blanton £ 
Alleyf 1975) discussed ear- 
lier is sufficient to show 
that the" increased emphasis 
on measurementf evaluation 
design* and theory* and the 
Introduction of critical 
benchmarks* have resulted in 
a more cumbersome tool* 
Unfortunately* each compo- 
nent of the model seems at 
oresent to be useful and 
desirable in an effort such 
as the School Action Effec- 
tiveness Study* Neverthe- 
less* this increased com- 
plexity suggests that a more 
streamlined model is appro- 
priate when doing short-term 
organizational development 
interventions* Just learn- 
ing the meaning of all the 
terms involved in the PDE 
structure is a large task* 
Consequently* for many brief 
organization development 
interventions* the selective 
use of those portions of the 
POE structure that seem to 
be the most relevant for the 
problem at hand is more 
appropriate than attempting 
to use all parts of the 
model * 
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P racti cal AQ pl i ca .^| On 

Summar i z i ng exper i ence 
with the use of the POE 
model is difficult* Because 
the model was created for 
this evaluation? experience 
in its use is limited* Tes* 
timonial evidence suggests 
that one or another part of - 
this process is useful to 
project imp 1 ementers in 
defining their own jobs? in 
formulating plans? and in 
clarifying their intentions* 
Testimonial evidence also 
suggests that the entire 
process is sometimes viewed 
as burdensome* On the 
whole? this structure seems 
a clear improvement over 
some more traditicnial evalu- 
ations because i t T^nvol ves 
implementers in evaluation 
planning? because it expli- 
citly attempts to build sum- 
mative evaluation structures 
based on an organization's 
forcefield? and because it 
focuses on goals and objec- 
tives of concern to imple* 
menters* 

The greatest virtues of 
the PDE model appear to be 
(a) its ability to elicit 
clear statements of the 
theory of action underlying 
a project? (b) its ability 
to elaborate clear measura- 
ble intervening outcomes? or 
objectives^ useful in 
assessing the effectiveness 
of interventions? (c) its 
ability to provide project 
implementers with the tools 
to assess their own efforts 
by measuring interventions 
against theory and objec- 
tives? (d) its ability to 
generate st r ateg i es per- 
ceived as feasible to imple- 
menters based on the diver- 
gent thi nk i ng that takes 



pi ece ^n forcef iel d anal y- 
sis? (e) its ability to 
involve project implementers 
in the eval uati on enterpr i se 
by engaging them in the spe- 
cification of measurabi e 
goals and objectives? and in 
the creation of evaluation 
des igns? ( f ) its ab i 1 i ty to 
provide short-term assess- 
ments of progress through 
the monitoring of critical 
benchmarks and tasks? 

(g) its ability to enable 
evaluators to understand the 
nature of a project by 

tr an si at i ng i mpl ementer *s 
ideas into a structured Ian-- 
quage of action research?and 

(h) its ability t3 serve as 
a structure for commun ica- 
tion between researchers and 
practiti oners that to some 
extent enables practitioners 
to becoine researchers by 
engaging in the study of 
their own actions* 

To some degree POE makes 
r i gor ous eval uati on more 
att racti ve to i mpl emen ting 
organ i zations despite i ts 
inability to make it truly 
pal at abl e to al 1 of them* 
Ideally? practice and evalu- 
ation would be merged, into a 
single enterprise in which 
rigorous research becomes an 
integral component of pro- 
gram operation* It is 
unrealistic to think that 
most practioners will ever 
acquire all of the technical 
skills required to systemat- 
ically conduct rigorous 
research on their activities 
(just as in is unrealistic 
to expect most researchers 
to biecome adroit practition- 
ers}* In addition^ truely 
r i gor ous research i s not 
always called for in the 
devel ooment of a project* 
When rigorous evaluation is* 
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called fort however? the PD6 
structure involving the col* 
laboration of researchers 
and impJ ementer s appearis 
helpful* Program Develop- 
jfient Evaluation does not 



successfully resolve many of 
the sources of tension in 
Rierging action with evalua- 
tionf and it is a taxing 
procedure for the evaluatort 
but it is progress. 
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Footnotes 



!• This use differs from some other common uses of the 
word object ive^-espec ial 1 y the usaqe of this term to imply a 
more specific restatement of a goal • A goal must itself be 
stated explicitly and measurably,* Diffuse statements of 
general or d i f f icu 1 t-to-spec i f y aims might be called mis- 
sions? but such vague aims are not to be confused with the 
crispf clear statements sought when specifying goals and 
objectives* 

2* Sometimes? a theory may imply a change in organiza- 
tional structure as an objective? but objectives stated in 
structural terms should be carefully scrutinized to make 
sure they are not statements of interventions* Iff .for 
example? the government of a country were experiencing major 
civil unrest? it may assume that the problem is due to a 
lack of respect for government* It might then impose mar- 
tial law in an attempt to restore civil order* Martial law? 
a structural change? might neither establish respect for 
government (an objective) nor civil order* If the estab- 
lishment of martial law were viewed as an objective rather 
than as an intervention? useful information would be lost* 
Viewing martial law as an intervention is more useful* 
Similarly? the revolutionaries in this same country may want 
to overthrow the government (an intervention) presumably to 
achieve some objective? such as freedom? more equitable dis- 
tribution of wealth? or a more satisfactory relation of 
workers to the mode of production* The distinction between 
a revolt and its objectives is an important one* Success- 
fully implemented revolutions do not always increase free* 
dom* 

Accordingly the POE structure makes a distinction between 
the objectives an intervention is intended to achi-^ve and 
the intervention itself* 

i 

3* Forcefield analysis is useful to evaluators and proj- 
ect managers for an additional reason* tapping the history 
of changes in the forcefield provides insight of potential 
general utility in planninn future projects* 

Strategies are used here as Lewin {19^7) used the term 
••over-al I olan»** Such an overall plan is composed of steps 
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taken in succession* the consequences of each of which is in 
principle discoverable* 

5. The concept of critical benchmark t«as incorporated in 
the PDE model at the suggestion of J. Douglas Grant. A cri- 
tical benchmark is what Lewin (1947) called a "gate." "The 
constellation of the forces before and after the gate region 
are decisively different in such a way that the passing or 
not passing of a unit through the whole channel depends to a 

l^high degree on what happens in the gate region (p. 332)." 

6. Tasks are related to what Lewin (1947) termed "action 
steps*" but differ slightly as used here. The PD6 model 
recognizes that a succession of tasks* any one of which may 
be fairly trivial in nature* are necessary to achieve some 
critical benchmarks. Often* however* strategies are suffi- 
ciently complex and the management of their execution suffi- 
ciently problematical that the specification of a number of 
tasks or "action steps" is a useful planning tool. 
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Figure 1 
The Program Development Model 
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Figure 2 

The Program Development Evaluation 
Model 

I 
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Table 1 



PROGRAM DEVELOPMENT EVALUATION CONCEPTS 



PROBLEMS: WHAT ARE THE MAJOR PROBLEMS, AND WHICH OF THESE ARE YOU ATTEMPTING 
TO ADDRESS? 



GOALS: WHAT LONG-RANGE GOALS IS YOUR PROGRAM DESIGNED TO REACH? 

(a) HOW CAN YOU MEASURE EACH GOAL? (HOW WILL YOU KNOW WHEN YOU HAVE 
REACHED EACH GOAL? 

(b) WHEN DO YOU EXPECT TO HAVE MADE A SUBSTANTIAL DIFFERENCE? 



(c) HOW WILIL YOU KNOW YOUR PROGRAM MADE THE DIFFERENCE? 



THEORY: WHY DO THE PROBLEMS EXIST? 



OBJECnVES: WHAT MEASURABLE CHANGES IN BEHAVIOR, AHITUDE, OR ORGANIZATIONAL 
STRUCTURE DO YOU EXPECT TO BRING ABOUT? 

(a) HOW CAN YOU MEASURE EACH OBJECTIVE? 

(b) WHEN DO YOU EXPECT TO HAVE MADE A SUBSTANTIAL DIFFERENCE? 



(c) HOW WILL YOU KNOW YOUR PROGRAM MADE THE DIFFERENCE? 



INTERVENTIONS: WHAT ARE THE MAJOR PROGRAM COMPONENTS DESIGNED JO ACHIEVE 
THESE OBJECTIVES? 



FORCE FIELD ANALYSIS: WHAT RESOURCES DO YOU HAVE AVAILABLE TO' IMPLEMENT 
YOUR INTERVENTIONS? 

WHAT OBSTACLES DO YOU ANTICIPATE ENCOUNTERING IN 

THE If4PLEMENTATI0N OF YOUR INTERVENTIONS? 



CRITICAL BENCHMARKS: WHAT SPECIFIC STRATEGIES: WHAT DEVELOPMENTAL 
MAJOR CHANGES MUST OCCUR TO STEPS WILL YOU TAKE? 

IMPLEMENT YOUR INTERVENTIONS, 
A'JD WHEN MUST THEY OCCUR? 
(What needs to happen by v/hen 
to get your interventions 
implemented?) 



TASKS: WHO WILL DO WHAT BY WHEN? 
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Measures 



Students and Teachers in Context: The Measures used in the 
school Action Effectiveness Study 

Gary 0- Gottfredsont Deborah K. Oqawat Donald E. Rick- 
ertf Jr^f and Denise Gottfredson 



Measurement is a central 
component of sound program 
development efforts* and 
measurement is essential in 
program evaluation* This 
report is a guide to using 
and interpreting measures of 
school climate and indivi- 
dual social development that 
may have broad applicability 
in school improvement 
efforts* It serves as a 
manual to help school admin- 
istrator st counselor St psy- 
chologistSf teacherst and 
school boards interpret sur- 
vey infcrmf^tion about school 
climates and about the char- 
acteristics of students and 
teacher s« 

Every good administrator 
has some way of taking the 
pulse of his or her organi- 
zation — of sensing when 
things are going wel 1 t when 
progress is being madet and 
of detecting problems or 



We are grateful for the com- 
ments of Michael Cook on a 
draft of this chapter^ The 
development of this report 
was sponsored in part by 
Grant No* NIE-G-80-0113 from 
the National Institute of 
Educationt U*S* Department 
of Education* The opinions 
expressed are solely the 
authors* f and do not neces- 
sarily reflect the positions 
or policies of any agency* 



areas where change is 
needed* Measurement is a 
technique for making that 
kind of assessment* Class- 
room teachers routinely give 
tests to measure their stu-* 
dents* progress in various 
currlcul ar areas* This 
guides them in moving on to 
new curriculumt in re-empha- 
sizing weak areasf and in 
meeting the educational 
needs of individual stu- 
dents* On a larger scalet 
economists routinely measure 
consumer prices and unem- 
ployment rates to help moni- 
tor the state of the econ- 
omy * 

In educationt measurement 
has traditionally focused 
mostly on student abilityt 
achievement f and interests* 
Educational deci s i on-maker s 
now have a large tool-kit of 
instruments to make measure- 
ments in these areas* These 
types of educat ional meas- 
urement have become so fam- 
iliar to us that we now take 
them largely for granted* 
They help in making scores 
of decisions related to 
counsel ingt placement? and 
instruction? and are regular 
feati?res in the landscape of ^ 
educational eval uat i on* 

These well-worn tools are 
limited? however? to a few 
areas* In other areas of 
concern to the education 
decision-maker? our measur- 
ing tools are less well 



-75- 

91 



Heasures 



developed* In some areast 
such as psychosocial devel* 
opmentf there are a variety 
of measuring devices availa- 
ble? but they are at present 
used primarily by the educa- 
tional or ps ychol og i cal 
researcher* In this area we 
havet for example? Holland 
and Baird^s (1968) Interper- 
sonal Competency Seal e? the 
Greenberger ? Campbel 1 ? Sor- 
ensent and □♦Connor t J« 
(1971) Psychosocial Maturity 
Inventoryt a variety of 
sel f -•esteem scales? several 
measures of i nternal -exter- 
nal control f and other simi- 
lar measures* All of these 
devices are potentially use- 
ful to the educational deci- 
sion-maker in assessing 
interventions aimed at the 
specific characteristics of 
individuals that they pur- 
portedly measure* 

efforts have also been 
made to develop measures of 
school or classroom climate* 
Perhaps the best known exam- 
ples of climate measures in 
the educational area are the 
commerci al 1 y avci lab! e 
scales produced by Moos and 
Trickett (1974). These 
scales are intended to meas- 
ure structure? orderliness? 
and so forth* Unfortu- 
nately? they suffer from 
techn ical psyc home trie limi- 
tat i ons (R i chards? 1978 ) ? 
which result in needlessly 
confounding the measurement 
of environments with the 
measurement of i ndi vi dual 
differences within the envi- 
ronment* Furthermore? they 
are insufficiently compre- 
hensive for the present pur- 
poses* But the measurement 
of school cl imates can now 
be improved by building on 



the research of Moos (1976)? 
Stern (1970)? Astin and Hol- 
land (1961)? and Gottfredson 
and Oaiger (1979)? as well 
as the practical work of Fox 
and assoc iates ( 1974) • 

A comprehensive and prac- 
tically oriented set of cli- 
mate measures can be of 
great value to educational 
dec i s ion- makers by making 
organ i zat ional di agnos i s 
feasible* A comprehensive 
diagnosis of this type is 
useful in organizational 
sel f-study and for program 
pi ann ing because it can 
point out the strengths and 
weaknesses of a school^s 
climate* And? a comprehen- 
sive climate assessment is 
of/value in assessing pro- 
gress toward improv ing the 
c I i mate? in detecting 
changes in the climate due 
to "naturally occurring" 
events? and evaluating 
school improvement efforts* 

Me a s ur i n g ^ I n d i v j j ^ §l^„gO ^ 

This chapter describes a 
two-t i er ed set of measures 
devised to meet the needs of 
those educational decision- 
makers who seek to i mprove 
educati on for i ndi v i dual 
students? or to improve 
school cl imate more gener- 
ally* One tier assesses the 
character istics of indivi- 
dual students and i ndi vidual 
teachers that are relevant 
to organizational cl imate? 
or to important per son a 1 
outcomes* The other con- 
sists of school-level cli- 
mate measures that directly 
assess some i mpor tant dimen- 
sions in which schools vary* 
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The measures are divided 
into these two classifica- 
tions for an important rea- 
son* We have all experi- 
enced differences in the 
psychosocial climates of 
different or gan i zat i onst and 
we can easily appreciate 
that organizations differ in 
the environments thst they 
provide* Yet we also know 
that different individuals 
often have different views 
of the characteristics of 
the same organi zat ion« 
Thereforet in assessing a 
given climatet it is impor- 
tant to average across many 
different reports-in essence 
treating individual differ- 
ences as error* These dif- 
ferences aret however? the 
very reason we measure indi- 
viduals* Accordingiyt two 
distinct sets of measures 
are called for* Besides the 
generalized assessmentst 
individual measures are 
needed for personalizing 
instruction and for compar- 
ing the effectiveness of 
al ter native educat ional 
tr f.'atments* 

The measures described 
here were developed specifi- 
cally for the School Action 
Effectiveness Study (SAES) 
because no comprehensive and 
psychometr ical ly adequate 
battery was available else- 
where* They are rooted 
directly in a program of 
research on delinquency and 
school env i ronments con- 
ducted over the past several 
years at the Center for 
Social Organ izati on of 
Schools* The instruments 
used are ^ased in part on 
those used in the National 
Institute of Education's 
(1978) Safe School Studyt 



instruments suggested by Fox 
and associates (1974)f the 
School Initiative Evaluation 
qjestionnairesf and a number 
of other instruments used i n 
major social surveys in 
recent years* In addition* 
relevant items (with neces- 
sary modifications) from 
other devices are used* 
Decisions about useful meas- 
ures are based on a review 
of the goals and objectives 
of the particular alterna- 
tive education project being 
eval uatedf on Gottf r edson* s 
(19Blb) account of delin- 
quency theory and strategies 
for organ i zat ional change t 
and on many discussions with 
action project person- 
nel — using the Program 
Development Evaluation 
f ramework--of the goals and 
objectives of their particu- 
lar school change efforts* 

Some^Essentiai^Psj^chometri^ 

In order to use the meas- 
ures about to be Jescritied 
in an informed manner? it is 
important to understand sev- 
eral ideas: (a) the rela- 
tive nature of psychosocial 
measurement f (b) rel iabil ity 
and (c) construct valid- 
ity*<I> The following para- 
graphs review these ideas* 

We have relatively few 
absolute measures in social 
science* Tn other wordst 
Simple counts of "units" of 
achievement or interpersonal 
competency or fairness are 
impossible to obtain* 
Insteadf we typically 
express their levels in 
relative terms* For exam- 
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plet achievement test 
results are often presented 
in terms of percentile rank 
or standard score form* 
These forms of expression 
involve statements of the 
standing of an individual 
(or organization) relative 
to some norm group of people 
(or isrgani nations)* For 
example? a percentile rank 
of 76 on an individual test 
would mean that out of 100 
indi vidual s representative 
of the population on which 
the test's norms are based? 
76 persons would have a 
score lower than this one« 

Somet i me s f toot some farm 
of ''scaled score** or ••stand- 
ard score** is used* Oftent 
raw scores are converted to 
rescaled scores with a mean 
of 50 and a standard devia** 
tipn of lOm (The mean is 
the arithmetic average of a 
set of scoresf and a stand- 
ard dev i at i on is a unit of 
dispersion or spread*) This 
way of expressing scores is 
especially useful when it 
turns out that scores have 
the familiar bell-shaped 
distribution; that is when 
lots of people (or organiza** 
tions) have scores near the 
middle of the distributiont 
and the rel at i ve frequency 
of scores trails off symmet- 
rically for higher and lo*ier 
scores* (Roughlyt this is 
what is meant by a ••normal •• 
distribution*} This kind of 
standard score is easy to 
interprett because about 68t 
of all persons (or organiza- 
tions) in the sample used to 
construct the norms have 
scores between 40 and 60* 
Figure 1 helps show how 
easily standard scores of 
this kind can be inter-* 



preted* About 951; of al 1 
scores f al 1 between 30 and 
70* Scores below 30 are 
relatively rare? and scores 
above 70 are also relatively 
rare* Comparati vel yv such 
scores are exceptionally 
high or exceptionally low* 
In this report we will use 
both percentile ranks and 
standard scores to present 
results* These simple forms 
of presentation were chosen 
in part because they are 
familiar to anyone who has 
interpreted the Schol astic 
Aptitude Testf which reports 
standard scores with a mean 
of 500 and a standard devia- 
tion of lOOf and which also 
repor ts percent i 1 e ranks * 

In interpret i ng such 
scores it is important to 
bear in mind that they 
express scores relative to 
the norm group* Oi f f erent 
narm groups wol themselves 
differ somewhat in their 
means scores ^and also in 
their dispersion)* There- 
fore a score that ist for 
exampl et at the 65th percen- 
tile relative to one norm 
group could be at the 30th 
percenti le relative to 
another norm group* There 
is no such thing as a magi- 
cally ••correct'^ or even 
"most appr opr iate** norm 
gr oup* In th i s r eport t the 
norm group is the total 
popal at ion of stuJentst 
teachersf or school s 
involved in SAESt unless 
otherwise explicitly stated* 
Thereforet a ••high*' score is 
a high score relative to 
these groups* 

Not all of the attributes 
of individuals and organiza- 
tions that are i mportant to 
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measure are normally 
di str tbuted* Sometimes 
scores tend to pile up at 
the top or bottom of a 
scalet and gradually trail 
off towards the other end. 
Rough I y speak i ngt this is 
what is meant by a skewed 
distribution* In such a 
distributiont interpreta- 
tions based on the assump- 
tion of a normal distribu- 
tion can oe somewhat 
misleading* Therefore? we 
sometimes point out the 
skewness of a distribution 
to aid in interpretation* 
For examplet delinquent 
behavior tends to show a 
markedly skewed distribu- 
tion^ with many people 
reporting few delinquent 
acts (or earning ••low*' 
scores) f and a very few 
individuals reporting a 
great many delinquent acts* 
Scores pile up at the bottom 
of the scale* 

The school St and hence 
the students and teacherst 
involved in this program may 
be expected to differ from 
nationally representative 
samplings* In particulart 
one of the selection crite- 
rion for participation in 
this program was a credible 
indication that the problems 
of crimet dropping outt or 
nona ttendance ^jere great in 
these school s or thei r 
cities* In additiont 
i nner-c i ty» predomi nantl y 
minority schools are clearly 
over represented* We know 
from other research {e*g*f 
Gottfredson L Oaigert 1979; 
Gottfredsont Joffet L Gott- 
f redsont 1981 ) that such 
school s on average exper i- 
ence more disruption than 
other public schools in the 
nat i on* 



To provide some perspec- 
tive on the way this norm 
group may differ from a 
nationally representative 
sample of school st we have 
prepared Table I* It shows 
selected characteristics of 
the SAES sample and nearly 
nationally representative 
samples of youth. The SAES 
schools enroll much larger 
Spani sh- American and Black 
popul al^tiafvs than the propor-- 
tion of these ethnic groups 
in the nation* and a much 
lower percentage of students 
enrolled in SAES schools 
live in intact families* 
Although victimized them* 
selves to roughly the same 
extent as typical studentst 
these students engage in 
much more delinquent activ- 
ity than do typical youth* 

Please note that the psy- 
chometric use of the word 
••norms'* has little or noth- 
ing to do with some everyday 
language uses of the word* 
In everyday language we 
SDmetimes use ••norm* to mean 
an i deal or required stand- 
ard* 3o Derek may be well 
below the weight riorms for 
women of her height f but 
there appears to be general 
agreement that her physique 
is not otherwise ••substand- 
ard," SImi larlyt it is 
quite possible for a school 
to have students who show an 
••average^^ degree of satis- 
faction with school but who 
are rather uncomf or table — or 
who are average in reading 
achievement according to 
large city normsf but who do 
not read wel 1 at al 1 * In 
interpreting any particular 
resultSf readers shouli pro- 
bably consider both their 
own ••ideal" norms and the 
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••statistical norms 
presented here. 

Chancet s loppinesst 
ambiguityt temporal insta- 
bilityt and heterogeneity of 
meaning or interpretation 
can infl uence any measure* 
Measurements of the distance 
between Saltiniore and New 
York made by the odometers 
in a number of different 
cars would tend to agree 
pretty wellt but not per- 
fectly. They would have 
highf but not perfect? reli- 
ability* Reliability is a _ 
technical term used to 
describe the relative con- 
tributions of measurement 
error and ••true" score vari- 
ability to a scale or other 
measure* Technically? reli- 
ability — i-s— the proportion of 
the variance (a statistic 
summarizing variability) in 
a score that is not error. 
Because there are many ways 
of defining error? there are 
many ways of estimating 
reliability (Stanley? 1971)« 
The reliability coefficients 
reported in this manual 
(alpha) are based on the 
analysis of items adminis- 
tered on a single occasion 
and therefore exclude tempo- 
ral instability froi*?; the 
de f i n i t i on of error • They 
can be interpreted as an 
index of how well the scales 
measure whatever they meas- 
ure at a given point in 
time* 

Knowledge of the reli- 
ability of a test or other 
index is important because 
low reliability means that 
the device does not measure 
anything well* A high reli- 



ability means that the 
device measures somethi nqi* 
( What that somethi ng is? is 
what construct validity is 
al 1 about* ) Kel iabi 1 i ty 
coefficients can range from 

0 to 1*0* A reliability of 
1«0 is highf meaning that 
the score contains no error* 
Over the years practitioners 
have devel oped rul es of 
thuinb for acceptable levels 
of reliability for di f ferent 
purposes * In general ? it is 
not sound practice to use 
tests with reliabilities 
much below .7 or *8 for 

1 nd i vidual di agnos is? per- 
sonnel decisions? and so 
forth. When i n ter pr etat i on s 
of patterns or profiles are 
to be made? it is especially 
i mportant that reliability 
be this high? or higher. 

For evaluation purposes? 
lower levels of reliability 
of ;nedsurement at the indi- 
vidual level are acceaptaole 
and are sometimes to be pre- 
ferred? because of three 
related considerations* 
First? because the scores cf 
many individuals are usually 
averaged in an evaluation? 
deoenciable estimates of 
true-score means can be 
obt at ned even w i th r at her 
unrel iable individual meas- 
ures (see Stanley? 1971). 
Second? the longer the scale 
(i.e*? the more iteins)t the 
more reliable it is? other 
things oeing equal? but it 
is often difficult? time 
consuming? or costly to 
administer long scales* As 
an al ternat i ve? us i ng short 
scales with many persons 
ga i ns good est i mates of 
group means. Third? in an 
evaluation it is necessary 
to measure many things* 
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This is because action 
programs have many goal s and 
objecttvesf and because it 
is always wise to search for 
unanticipated positive out- 
comes or side-effects of a 
program. But administering 
many highly reliable (i.e.t 
long) scales is prohibitive. 
Fortunatelyt a large test 
gr oop aga in comes to the 
rescue. Using short scales 
with many people solves the 
problem and yields satisfac- 
tory estimates of true-score 
means. 

As a rule of thumbt 
scales with reliabilities as 
low as .5 are adequate for 
use in an evaluationt EL2* 
vide d that the project being 
eval uated uses r andonjxzati^on 
as a selection device? or 
that any selection is abso- 
lutely independent of (i.e.t 
unrelated to) the goals or 
objectives of the program. 
In such an evaluationt it is 
not necessary to attempt to 
adjust for pre-existing or 
spurious group differences 
on outcomes. When it is 
necessary to make such 
adjustments by using statis- 
tical "control Sf** rel iabil i- 
ties for the control varia- 
bles must be as high as 
possible* The rule of .5 is 
too lax in this case because 
when the "control* variables 
are unreliable they do an 
inadequate job of correcting 
for spurious differences 
between groups . Therefore t 
to enabi e a sound eval ua- 
tiont a project which does 
not randomize should use 
more reliable (i.e.t longer) 
scales encompassing measures 
of all relevant characteris- 
tics in which the treatment 
and comparison groups may 
di f f er • 



Validity 

Validity has to do with 
the meaning and interpreta- 
tion of an index or scoret 
and is closely linked with 
theory. Theory involves 
constructs or ideas about 
the causes or nature of phe- 
nomena. Of tent measurement 
has meaning only in the con- 
text of some theory* For 
examplet some educators have 
a theory that a general 
ability called intelligence 
underlies much human perfor- 
mancet or a least scholastic 
achievement. The measure- 
ment of intell igenceVusing a 
paper and pencil verbal 
ability test may make sense 
in terms of this theory. 
Because the theory predicts 
tnat this test will corre- 
late with school gradest 
evidence about the validity 
of a test for measuring the 
construct of intelligence 
can come from an examination 
of the empirical relation 
between test scores and 
school grades. The same 
evidence provides informa- 
tion about the utility of 
the theory. Theories and 
tests are thus validated in 
a common process. *^e speak 
of a test as validated when 
empirical evidence has in 
general shown the test 
results to follow the ore- 
dictions of 3 theory that 
has been usef ul . 

In additiont when there 
is agreement about what a 
construct meanst some evi- 
dence about validity can 
come from an examination of 
vJie item content of a test. 
For exampl et most of us 
would probably agree that a 
test to see how many bricks 
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a person can load on a truck 
in an hour is a poor test of 
verbal ability? and that a 
list of mul t iple-'choice 
vocabulary items would pro- 
vide a more valid measure of 
that construct. (Similarly? 
the vocabulary test would be 
a poor test of endurance*) 
Therefore? del iberately 
including items to measure a 
given construct in itself 
can provide some limited 
degree of confidence in a 
scalers construct validity* 

The evidence is strength- 
ened if the scale shows 
expected patterns of corre- 
lations with other scales* 
And it is especially 
strengthened if applicable 
exper i mental manipul at ions 
influence scores in pre-- 
dieted ways* Other evidence 
of validity can come from an 
examination of differences 
in scores on the scale among 
groups known or bel ieved to 
differ in the characteristic 
being measured* For widely 
used instruments? these 
kinds of evidence accumulate 
over time* Eventually? a 
basis for judgment about a 
scale's construct validity 
emerges--al though different 
judges often di sagree* 

There can therefore be no 
such thing as an absolutely 
validated test or scale* 

Subsequent sect i ons 
describe the origins? devel - 
opmenit? and psychometr ic 
properties of a multi-*level 
set of assessments of 
schools and their inhabi- 
tants* These sections are 
intended to provide informa- 
tion about reliability and 
validity? and to describe 



the normative interpretation 
of these as sessment s* 

?!gasures^of ^Students 

Five sets of measures of 
i nd i V idual students have 
been developed to measure (a) 
constructs suggested by the 
staff of action projects? 
and (b) delinquency preven-- 
tion theory* These measures 
of students are needed to 
assess project effectiveness 
under difficult field 
research conditions and to 
learn more about what works 
for whom* 

Social_8ackaround 

Measures of social back- 
ground or family character- 
istics are needed for two 
reasons: (a) They provide 
essential statistical con- 
trols to aid in demonstr at- 
i nq project effectiveness 
when eval uat i on des iqns 
calling for statistical 
adjustments are neces sary ? 
or when stronger designs 
f al 1 apart* (b) In a few 
cases? projects aim to alter 
family character i St ics--usu- 
al 1 y the extent to which 
parents value education or 
encourage their children to 
perform well in school* 

Accord i ngl y? the f ol low- 
ing six measures were devel- 
oped: 

Pa£ental^_E ducat ion* This 
two-item scale is based on 
decades of research that 
show parental education to 
be a powerful antecedent of 
school ing outcomes? espe- 
cially of persistence in 
education ( Sewel 1 ? Hal 1 er? C 
Portes? 1969a)* The two 
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items ask how much education 
a student's father and 
mother completed. The scale 
has a reliability coeffi- 
cient of .76 andf with the. 
exception of the two ethnic 
groups for whom only very 
small samples are availablet 
has approximately equal 
reliability for all race-sex 
subgroups examined* Table Z 
displays the scale's reli- 
ability estimates for ten 
groups. (At a later timet 
parental occupational level 
will be added to this scalet 
as much research (OuncaOf 
I96i; Treimant 1977) indi- 
cates is appropriate. 
Write-in data about parental 
occupation require much time 
ro process.) This measure 
may be taken as an indicator 
of family socio-economic 
status. It is known to be a 
good predictor of schooling 
outcomes such as persistence 
and grades (Bachmanf O'Mal- 
ley E JohnstoHf 1978; 
JenckSf 1979)f but it is 
usually only weakly related 
to delinquent behavior at 
the individual level — al- 
though perhaps it has a 
stronger relation to more 
serious delinquency (Tittle 
t Villemezf 1978; Gottfred- 
sont 1931a). 

P are n t al_E mghasi s _on_^ d u- 
catTon.~ This four- item 
scale asks for information 
about the degree of parental 
attention to the student's 
school performance and 
parental expectations for 
school persistence. It was 
suggested by action project 
theories that attributed 
student non-attendance to a 
1 ack of parental encourage- 
ment or "value" on educa- 
tion. Andf parental influ- 
ence is demonstrably 



predictive of student per- 
sistence in school (Ottot 
1976). The scale is only 
moderatelv rel i abl e-- 
overall* with coefficients 
ranging from .38 to .62 for 
race-sex subgroupSa Reli- 
ability coefficients are 
only slightly lower for 
black subsamples* where sin- 
gle-parent (female-headed) 
family structures make the 
potential differential util- 
ity of this scale a matter 
for concern. ?^odest differ- 
ences among groups existf 
witH American Indian males 
reporting rather low paren- 
tal emphasis on education 
relative to other groups. 
(See Table 3). The scale 
has moderate negative corre- 
lations with self-reported 
delinquency (see Table 4)f 
and has an expectedf but 
small* positive correlation 
with student reports of 
effort spent on school work. 

Hater nal3ole_Modei_i!;ie3- 
ativei. "ATthough a sensi- 
tive"matter to address in a 
survey used in schools — es- 
pecially where administra- 
tors are uncertain of the 
confidence of their communi- 
ties—this scale is intended 
to tap an important theoret- 
ical antecedent of delin- 
quent behavior. Social 
learning theory (aandura. 
1971) and differential asso- 
ciation theory (Sutherland t 
Cressey* 1955) appear to 
imply that a person learns 
to behave in accord with 
models in the person's envi- 
ronment. This five-item 
scale contains items asking 
whether the person's mother 
gets mad a lot* drinks too 
much* or spends all her 
money on herself* It is 
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characterized by loi^ item 
responses (i«e*f wost stu- 
dents say "no*' to most 
items) f and therefore has a 
low rel i abi 1 i t y~« 36 over- 
allf with male white and 
American Indian reliabili** 
ties very low« Despite its 
low reliabilityt it corre- 
lates mZZ with self-reported 
del i nquent behav i or t mak i ng 
it a potentially useful sta- 
tistical control var iabl e in 
weak evaluation designs* 

?!?.gter naj^Role Mode 1 
(Pgs it i ve] » This scaTe is 
composed of items describing 
positive aspects of the 
maternal role model* These 
items were included largely 
to soften the impact of the 
items in the negative role- 
model scale just described* 
The scale has a modest to 
dismal degree of reliability 
(especially for white and 
Indian males)? and this set 
of it^ms 1 mother is a hard 
worker? fixes things around 
the house? etc*) is rela- 
tively uncorrelated with the 
Negative Maternal Role Model 
Seal e* 

at i ve) * This scale paral- 
lels the cor respond i ng 
Maternal Role Model Scale* 
It has soafiewhat higher reli- 
abilities than that scale? 
and is moderately correlated 
with sel.f-r epor ted del in- 
quen t behav i or * 

Pa te rnal Rglg, Models 
H^Pos i ti ve| * This scale oar- 
allels the cor respondi ng 
maternal scale in intent and 
in psychometric characteris- 
tics* It is only modestly 
correlated with the corres- 
ponding maternal scale* 



Table 5 shows correla- 
tions among the family back- 
ground seal es * These corr e- 
lations are modest^ and low 
relative to the scales' 
reliabilities--imp lying that 
the scales each measure a 
relatively independent 
dimension of family social 
background* 

Three measures of a stu- 
dent • s social relations were 
devel oped bee ause of 

(a) empirical and theoreti- 
cal links between bonds of 
affection or respect for 
others and conforming (non- 
del inquent) behavior? 

(b) power ful statistical 
assoc i ati ons between del i n- 
quent behavior and delin- 
quent peer infl uence? 

(c) the central place given 
to peer influence in the 
theories of several of the 
act i on pro j ec ts ? and ( d ) the 
expl icit assumc>tion made by 
several projects that paren- 
tal supervision governs stu- 
dent attendance * One of 
these seal es (attachment to 
parents)' is intended for use 
as a statistical control 
variable to strengthen eval- 
u at ion desi gns* The others 
measu re outcome s of i mpor- 
tance for all or some proj- 
ects* 

Attachment to Parents* 
Thi s seal e? i nteoded to 
measure H i r schi • s construct 
of the same name? i ncorpo- 
rates several i terns closely 
related to items shown in 
earlier studies to be corre- 
lated with delinquent behav- 
ior (Hirschi? 1969; Hinde- 
langt Hirschif t Weis? 1981; 
D* Gottfredson? 1981b)* An 
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attempt has been made to 
engineer a more potent scale 
by includiny more items 
related to this construct. 
The six-item scalet asking 
students tyoi^ close they are 
to their parents? how much 
they like themt and so 
forthf has an overall reli- 
ability of .61. It corre- 
lates as e)f}>ected with 
self-reported delinquent 
behavior (see Table in 
accord with Hirschi's (1969) 
theory that attachment to 
parents creates a stake in 
conforming behavior. T^^s 
agreement provides some evi- 
dence of the construct val- 
idity of this scale. 

This^scale measures a con- 
struct central to the expla- 
nations of delinquency and 
non-attendance formulated by 
several of the action proj- 
ects. It is rooted directly 
in earlier research (summa- 
rized by cmpeyt 1978) that 
shows delinquent peer asso- 
ciations to be powerful ore- 
dictors of delinquent 
involvement. In additiont 
it incorporates items 
related to cropoutt similar 
to those used in earlier 
studies of persistence in 
schooling (Bachman et a'^.t 
1978). It iSf howevert an 
attempt to engineer a 
longert more powerful t and 
broader-based- measure of 
negative peer influence. 
This nine-item scale has 
reU3bilties ranging from 
•53 to .7^ across subgroups 
(Table 2) andt as expectedt 
is a potent correlate of 
delinquent behavior (Table 
A ) • It conta i ns items ask- 
ing whether the student* s 
best friend is interested in 



schoolf thinks getting gooo 
grades is importantt thinks 
school is a paint or has 
been involved in delinquent 
act ivi ties. 

Although we know of no clear 
evidence that parental 
supervision is related to 
delinquencyt at least two of 
the action projects attri- 
bute truancy and nonatten- 
dance in part to 3 lack of 
parental supervision. 
Thereforet we attempted to 
build a scale tapping stu- 
dent reports on whether 
their parents usually know 
where they are and what they 
are doingt and whether as 
far as their parents are 
concerned they are free to 
come and go as they choose. 
This scalet with only two 
itemst has low reliability. 
It doest howevert correlate 
with delinquent behavior and 
with students* efforts in 
school t as predicted by the 
action project theories 
(Table ^)« 

The correlations imong 
these measures of social 
relations are presented in 
Table 6. It shows that 
parental supervisiont which 
forms only a weak scalet is 
correlated as expected with 
the other measurest but that 
these correlations are close 
to its reliabilityt raisinq 
doubts about the usefulness 
of maintaining this scale as 
a separate entity. Its cor- 
relation with delinquent 
behavior is so hi;jht how- 
evert that the measure is of 
some value as a statistical 
control • This scale should 
be improved if possible. 
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Att i tudes and Soc ia|,^D eye 1 
Q pment 

Social development is a 
major goal of the Al terna* 
ttve Education Program. In 
this areat there was consid-- 
erable prior work to build 
on in choosing measures to 
include in the battery* 

^liSQ^t iSO • four- 
item Alienation Scale is 
based in part on Srole's 
(1956) Anomia Scalet but 
fewer items are included^ 
and the wording of itmes has 
been changed to give them 
more school-related coiitent 
and to make them sound a 
little less bizarre. Speci- 
ficall/f alienation items 
used in the School Initia* 
tive Evaluation (Grant et 
al«f 1979) were modified for 
use here* Items include^ 
••These days I get the feel- 
ing that I*m just not a part 
of things^** Andt I feel no 
one really cares much about 
what happens to me*** Over-* 
all f this short scale has a 
reliability of m^^f and 
works about equally well for 
all groups examined except 
for the very small sample of 
Asian malesf who score quite 
low on the average* As 
expectedf the scale corre- 
lates positively with self- 
reported del inquent behav 
iort and negatively with 
reports of effort expended 
on school work (see Table 
^)« 

Atta chm ent^ to School • 
This is a central construct 
for many projects whtise 
major goal or objective is 
the development of positive 
student atti tudes toward 
school* The construct is 



also central to social 
control theor ies of delin- 
quency (Hirschi ? 1969) that 
view attachment to school as 
a major social bond 
restraini ng individuals from 
participation in del inquant 
behav i or • Consequentl y t we 
have constructed a rela- 
tively long and !5road-based 
measure of attachment to 
school* This lO-item scale 
has reliabilities ranging 
from *67 to *81 across sub- 
groups — *75 overall. Items 
ask the students if they 
like the school? if they 
1 i ke the cl asses t how i mpor- 
tant getting good grades 
aret and so forth* The 
scale isf as expected? a 
powerful correlate of delin- 
quent behav ior (negative) 
and effort expended at 
school work (positive)* 
Other cor rel a t i ons ( not 
shown in a table) indicate 
that attachment to school is 
also related to school 
attendance* 

Belief* The expectation 
that individuals differ in 
the extent to which they 
bel ieve in the moral val id- 
it y of c onven t i onal soc i al 
r ul es f and that the degree 
of belief influences behav- 
ior t is widely shared* A 
common goal of pear-group- 
based interventions to pre- 
vent delinquency is to 
strengthen belief by using 
peer pres sure * The i tern 
content of Gough's (1964) 
soc i al irat i on seal e-- which 
was devel oped through emp i r- 
ical efforts to discriminate 
between of fenders and non- 
offenders — I ends support to 
thji^ popu I ar not i on* Andt 
belief is a central con- 
struct in social control 
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theoryt which postulates 
that people differ in the 
degree to which they have 
internal izea rulest and that 
they therefore are con- 
strained from Snvolveroent in 
delinquent behavior to dif- 
ferent degrees. Much empir- 
ical evidence supports this 
idea (e.g«f Wiatrowski G 
SwatkOf 1980; Gottfred- 
sont l9Blb; Hirschif 1969) • 

Consequent! yt in order to 
measure this aspect of 
social development we have 
assembled a short scale from 
well-worn items used in 
other surveys f whose charac- 
teristics were known. The 
six-item scale contains 
items such ast "It is all 
right to get around the law 
if you can;" "Taking things 
from stores doesn't hurt 
anyone;" and "People who 
leave things around deserve 
it if their things get 
taken." The scale has a 
reliability of about •SOt 
and it correlates .27 with 
sel f-reported del inquent 
behavior? as earlier 
research and theory prec?ict 
it should. 

This scale is composed of 
four items from Holland and 
Baird's (1968) Interpersonal 
Competency Scale. That 
scale has well-studied psy- 
cnometric properties. It 
consistently has moderate 
rel iabi I i ty and correlates 
positively with other meas- 
ures of psychological heal th 
or adjustment? ^and nega- 
tively with measures of 
alienation. The fifth item 
was written by Holland espe- 
cially for the present pur- 
pose? to gi ve the scale more 
school-related content. It 



has a reliability coeffi- 
cient overall of This 
measure of social develop- 
ment correlates positively 
with reported ef f or t 
expended on school work? and 
it is nearly independent 
(uncorrelated with) self-re- 
ported :lel inquent behavior. 
This accords with other evi- 
dence that delinquent 
involvement is only modestly 
associated with psychologi- 
cal health (Waldo & Oinitz? 
1967)? although there are 
some alternative views 
( e.g» ? Ciuay ? 1964) • 

I nvol^vemen t. This seal e 
is Intended to measure a 
central construct in social 
control theory that does not 
appear to have been wall 
measured in the past. The 
idea is that involvement in 
conventional act ivi ties 
creates a stake^ in conform- 
ity? because a person 
involved in rewarding activ- 
ities has something to lose 
by misconduct. This scale 
(not to be confused with 
environmental measures of 
student influence or 
involvement in dec i s i on-diak- 
ing) is composed of 15 it:?ms 
(most of which were adapted 
from the current National 
Longitudinal Study question- 
naire) asking about a stu- 
dent's participation in a 
wide variety of in-school 
and out-of -school activi- 
ties? including school-spon- 
SDred activities and work. 
It has an overall reliabil- 
ity of .75? but does not 
correlate as expected with 
reports of delinquent behav- 
ior? casting some doubt on 
its construct validity^ It 
does? however? correlate .31 
with students* reports of 
rewards they have received 



-87- 

103 



Measures 

in schoolf and has a small 
• 12 correlation with a nieas- 
ure of practical knowledge 
(to be described shortly)* 
Although this scale was 
intended to serve as an 
important intermediary out- 
come measuret its utility is 
in doubt* 

Posi tiyg Sglf-ConcgBt • A 
number of sel f- esteem seal es 
with wel 1 -researched proper- 
ties (Robinson L Shavert 
1973f review more than 30 
measures) are available* To 
create a short scalet items 
previously used by Rosenberg 
(1965) and an item similar 
to one used by Coopersmith 
(1967) were subjected to 
analysis along with another 
set of items constructed to 
capture aspects of self-con- 
cept specific to schooling 
and delinquency* This scale 
has its base in the labeling 
perspective (Lemmertv 1972)f 
which implies that if people 
are treated as slow learners 
or delinquentSf they will 
come to incorporate aspects 
of those social definitions 
into their own self-con- 
cepts* Positive self-con- 
ceptf therefore* is an 
important intermediary out- 
come according to this per- 
spective* Effective alter- 
native education projects 
would presumably increase 
scores on the positive 
self-concept scalet and a 
program with unexpected neg- 
ative side-effects could 
decrease scores* Item anal- 
ysis did not justify treat- 
ing self-esteem as a sepa- 
rate scale from these 
1 abel ing outcomes* because 
items are about equally CDr- 
related across the two sets* 
Weak items were excluded* 



1 eavi ng a 1 2- i teifi seal e wi th 
reliabilities ranging from 
•51 to *T8 across subgroups* 
•63 overall • (In these as 
i n other i tem anal yses* sta- 
tistics for all sut)groups 
were examined to ensure that 
i tems worked gener a 1 1 y 
across groups • ) I tems 
include* "My teachers think 
I am a slow 1 earner;" "Some- 
times I think I am no good 
at all;" "T am the kind of 
person who will always be 
able to make it if I try;" 
and "I do not mind stealing 
from someone- -th=»t is just 
the k ind of person I am* " 
The scale correlates *^8 
wi th reported effort on 
school work* and -*24 with 
sel f- reported del i nquency * 
It also correlates -*39 with 
alienation and * 39 with 
i nter per sonal competency* 
lending support to its con- 
struct val i di ty* 

Pr act i_c a l^_Knowl_edge* To 
provide a simple measure of 
sel f-report ed competenc i es 
needed for coping with ever- 
yday life* a seven-item 
measure was created for the 
survey* A 1 though thi s 
self-report scale may be a 
p3or subs t i tut e for a more 
comprehensive or task sample 
approach* it seemed the only 
way to build a measure of 
this kind of soci al devel op- 
men t i nto a mul t i -pur pose 
battery* The scale works 
remarkably well* with a 
r el i abi 1 i ty coef f i c i en t of 
• 71* and good i tem proper- 
ties across all groups stu- 
died* It is r el ative iy 
i ndependent of the other 
measures of att i t Jdes and 
social development* and of 
sel f-reported del i nquent 
behavior* Because it has 
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not been well-studiedf it 
should be interpreted cau- 
tiously. We will be able to 
report more information 
about this scale after otner 
data are merged with the 
quest ionnai re measures* 

Rebel 1 ious_^Autgnomjt^ ^ " 
talking wi th^'persons running 
action projectsf especially 
the Peer Culture Development 
Project in Chicago* explana- 
tions of the problem of 
del inquency sometimes 
involved a kind of peer or 
gang culture that resembles 
Miller's (I95b) characteri- 
zation of subcultural 
socialization. The peer or 
gang culture may incorporate 
a set of socially-shared 
expectations that are dif- 
ferent from what might be 
called middle-class expecta- 
tions. Differences may be 
so great that in behaving 
according to the "lower- 
class!* system a person may 
violate norms of middle- 
class culture* and may 
appear to be deliberately 
non-conforming or malicious 
to a "middle-class" obser- 
ver. In p3/rticular* middle 
class concerns with achieve- 
ment may not be shared by 
••lower class" youth (cf# 
Attachment to School and 
Educational Expectations) ^ 
Instead* these "lower-class" 
youths* according to Miller 
are concerned with trouble* 
toughnessf smartness ( i .e. * 
manipulative skill)* excite- 
ment* fate (explaining 
events by reference to 
chance or luck)* and auton- 
omy (an ambivalent relation 
to authority — overtly desir- 
ing not to be pushed around 
but covertly desiring to be. 
cared for and controlled). 



Because of this recurrent 
theme in our discussions 
with action project person- 
nel* it seemed important to 
incorporate brief measures 
of this type of "subcul- 
tural" value system. Unfor- 
tunately* we know of no dev- 
ices already in existence 
designed to measure this 
constellation. But we could 
locate isolated items from 
previous surveys whose dis- 
tributional properties could 
be examinedf and we found 
some interview quotes that 
suggested some items. We 
then wrote* therefore* nine 
items that seemed to capture 
the essence of thase themes. 
Item analyses implied that 
three of these items formed 
a scale for all race-sex 
subgroups. The deletion of 
poor items* howevert nar- 
rowed the content of the set 
down to items that appear to 
reflect a rebellious auton- 
omy: "i^hether or not I 
spend time on homework is my 
own business;" "I should not 
have to explain to anyone 
how I spend my money;" and 
«I don^t like anybody tell- 
ing me what to do." The 
scale has a reliability of 
.^6 overall. The scale cor- 
relates .18 with self-re- 
porced delinquent behavior 
and -.24 with bel ief . 

The correlations among 
the eight measures of atti- 
tudes and social development 
are shown in Table 7. These 
correlations indicate that 
Alienation is closely (nega- 
tively) related to Attach- 
ment to School. Indeed, the 
correlation between these 
two scales is higher than 
the rel iabil I ty of the 
former scale. This implies 
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that some parsimony would 
result from combining these 
scales* We report them 
separately only because we 
believe users will like to 
see them separately* We do 
not recommend making sub-* 
stantive interpretations of 
di f ferences between these 
two scales. The ether 
scales are all relatively 
i ndependentf impl y ing that 
they measure fairly distinct 
aspects of attitudes and 
soc i al devel opment • 

SeJ^^Regor ted^Behavior 

Sc hoo l _Ef for t« That stu- 
dents who earn low grades in 
school tend to drop out of 
school and to engage in 
delinquent behavior more 
than others are two of the 
best documented and consis- 
tent findings in the litera- 
ture (Gottf redsont i981a)« 
Social class and ability are 
modestly associated with 
these same outcomes but do 
not completely account for 
these assoc i at i onv>« There- 
foret it seems likely that 
these outcomes are deter- 
mined at least in part by 
grades--the majort if infre- 
quently appliedf reward sys- 
tem of traditional school- 
i ng« Grades in school are 
not determined solely by 
ability and social classt of 
course* Industrial psychol- 
ogy* s expectancy theory 
(Porter £ Lawlert 1968) sug- 
gests a mechanism whereby 
effort is expended if valued 
rewards are perceived as 
attainablet and in which 
effort is one of the deter- 
minants of Doth performance 
and rewards* Therefore 
effort is an important 
intermediary outcome varia-- 
ble that should be assessed 



in the evaluation of a pro- 
gram desicjaed to prevent 
delinquency and foster per- 
sistence in school ing. 

Unabl e to locate exi st i ng 
questionnaire measures of 
this construct? we developed 
orie« Tnis five-item scale 
has a reliability of ,51 
overall* (It is somewhat 
less rel i abl e for Spani 
Ainericans in this samp^et 
presumably because some of 
its item content deals with 
homework? whi ch is rare I y 
assigned in Puerto Rico 
where most of the Spanish 
Americans in the sample 
attend school*) The scale 
i ncl udes these i tems : ••Com- 
pared to. other students? how 
hard do you work in 
school ?•• • ••I turn my home- 
work in on time**; and ••! 
don* t bother wi th homework 
or class assi gnments*** As 
expect<3df f emal es score 
higher on this scale on the 
average than do males* It 
correlates .39 with self-re- 
ported grades and .34 with 
attachment to school? sup- 
porting its interpretation 
as a measure of effort 
expended on school work* 

The Alternative Education 
Program is intended to dem- 
onstrate and evaluate proj- 
ects that aim to increase 
attendance* Jependabl e 
attendance data are not 
always available from school 
records? so we decided to 
incorporate a brief self-re- 
port measure of attendance 
in the questionnaire to pro- 
vide back-up data* This 
decision proved to be wise: 
we were unabl e to obtain 
these data from records for 
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at least one project t the 
data from many others is so 
erratic or incomplete as to 
be of limited valuer and 
some attendance data that 
will ultinately prove valua- 
ble will require extensive 
editing before it is usable^ 

Two itemsf one asking how 
often the student cuts 
school all day and one ask^ 
ing about class skipping? 
compose this brief scale? 
with an overall reliability 
of #66« 

S eif2££22£t£^^£ii29iiS!2 9,1 
(Total ! • One way to find 
out what people do i s to ask 
them* Naturally? not every- 
one tells the truth? perhaps 
especially when the ques- 
tions are sensitive** A com- 
mon assumption is that peo^ 
pie will conceal information 
about their participation in 
illegal behavior? and so 
under-report» At the same 
time? the rates of delin- 
quent behavior estimated by 
the self-report method are 
higher than those derived 
from official records (£mpey 
t Erikson? I96b). There is 
thus a great deal of debate 
among cr i mi nol og i s t s about 
the appropriate way to meas- 
ure criminal behavior • 

Although there is no need 
to go into the arguments in 
any detail here? a major 
issue is that typical self- 
report measures (e«g«? Nye? 
1958) tend to measure minor 
"of f enses?** some of ^hich 
are not "crimes?" or would 
not be crimes if committed 
by an adult* Elliott and 
Ageton (1980) have recently 
presented evidence that 
self-report scales involving 



more serious offenses tend 
to resemble measures based 
on official data more than 
do scales involving only 
tr i vi al i terns • Hindel ang? 
Hirschi? and weis (1981) 
have also just published the 
results of a major examina- 
tion of self^report delin- 
quency jneasur ement • 

The bottom line? insofar 
as it can be perceived at 
present? is that fairly 
long? variety-typa scales 
involving a range of serious 
delinquent behavior do pro- 
duce results that parallel 
of f ic ial records f or _ soma 
subg r g ups but not for oth- 
ers. Hindelang et al« 
(1981) report validity coef- 
ficients for a number of 
variables that imply very 
low validity of self-re-^ 
ported data for officially 
"delinquent" black males? 
aad much better validity for 
other subgroups* This is a 
difficulty that should be 
kept in mind in interpreting 
these self-reported data* 
It appears related to a 
similar problem of differen- 
tial reliability in studies 
of educat ional per si stence 
(Bielby? Hauser? L Feather- 
man? 1977; 0. Gottfredson? 
1981a)? and it points out 
the importance of obtaining 
official data for pt*rposes 
of ev al uat i on* 

The specific self-report 
measures used here are modi- 
fied from those used by 
Elliott and Ageton (1980) 
and by Hindelang? Hirschi? 
and Weis (1981)* Many o*" 
Elliott^s items were wCied? 
but pretests showed thjt the 
response formats c reated 
problems iv^ group question- 
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naire administration* At 
about the same timet a manu- 
script of the Hindelang et 
al • book became available? 
in which the authors 
observed that "ever variety** 
and '•last-year variety" 
scales yielded very good 
resul ts (and a 
consul tant* s--La Mar 
Empey* s*-advice suggested 
the same)* For purposes of 
evaluationt "ever variety" 
items (which measure preva- 
lence rather than incidence) 
could nov. be used. There- 
fore we designed "last-year 
variety*^ items. These items 
askf "In the last^j^ear have 
you. Respondents'^Tndi - 

catet for example? whether 
they have "stolen or tried 
to steal something worth 
more than S50." 

A 19-item scale con- 
structed in this way has 
very nice characteristics-- 
-considering that only a 
small proportion of respon- 
dents answer yes to any 
given question. Overallt 
reliability is «84. In the 
hold-out sample— those not 
used to conduct item analy- 
ses? but set aside to obtain 
unbiased estimates in a new 
sample--the subgroup reli- 
abilities range from .63 to 
.93. The single low coeffi- 
cient is f or Asi an- Amer i can 
femaleSf who report almost 
no del i nquent behavi or . 
These reliabilities compare 
favorably to those obtained 
by Hindelang et al. (1981) 
with a 63-item last-year 
variety scale — .83 to .92 
for black and white males 
and females. 

Readers interested in a 
more thorough understanding 
of scales of this type and 



their relation to other 
variables of interest should 
see Hindelang et al . (1981). 

iSlIzise2ItSi3-.2L!i3_!Jse . 
Action project personneT" 
have shown cons i derable 
i nterest in a component of 
del inquent behavior involv- 
i ng drug use. To provide a 
measure to meet their needst 
we have al so scored a f i ve- 
item suDset of the longer 
(total) S-R delinquency 
scale. It is composed of 
items asking about the use 
of cigarettesf liquort mar i - 
juanat and other drugst and 
about going to school 
"high." {\ sixth item about 
glue sniffing was left out 
because the analyses did not 
support its inclusion for 
all ethn i c groups . ) This 
group of items closely 
resembles the Hindelang et 
al . (1981) Drug Index. It 
has an pverall reliability 
of .34. 

S eX f z 2. g B 2 £ i §d . 5 e r i o u s 
2§liD2iiSQ£y • ^ second""sub- 
scale was constructed to 
measure only conduct that 
nearl y everyone woul d regard 
as criminal. It includes 11 
i tems (including one about 
selling drugs that Hindelang 
et al. would pi ace in the 
drug cluster) and has an 
overall reliability of .790 

ences 

It is anticipated that 
tne projects in the Alterna- 
tive Education Program will 
expand the range of school 
rewards beyond those repre- 
sented by traditional class- 
room grades. Accordinglyt 
we have developed two scales 
to measure students* 
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rcMdrding and punishing 
experiences f in an effort to 
assess this important but 
hard-to-measure set of out- 
comes* School rewards and 
punishments make sense 
intuitively as probable 
causes of school attachment? 
effort? and persistence. 

School^P uQishmen t s • i s 
fou7~i tern"" scale Ts an index 
of the negative sanctions an 
individual student experi- 
ences. It asks whether the 
student was required to stay 
after school? given an extra 
assignment? or had his or 
her grade lowered as a pun- 
ishment. Its reliability 
coefficient for tne total 
sample is .54; according to 
this index males experience 
more punishment? as 
expected. The scale corre- 
lates .30 with self-reported 
delinquency? --28 with posi- 
tive self-concept? -.30 with 
belief? -.22 with school 
effort? and .24 with nega- | 
tive peer influence. 

S£t]22l-i£ii3Il^^ • ^^^^ 
six- item scale is an index 
of the positive sanctions an 
individual student experi- 
ences. It includes reports 
of incidents in which the 
teacher complimented the 
student's work? the student 
was given a prize or award? 
or the student won an award 
for his or her class. The 
reliability coefficient for 
the entire sample is ^S^. 
The scale is relatively 
independent of sex? and is 
correlated .25 with school 
attachment . 

Victimization. A final 
measure of school experi- 
ences deals with personal 
victimization. It is 



intended for use in assess- 
ing the amount of crime in 
the environment? 3nd it is 
used in the aggregate to 
characterize the school. 
Nevertheless? the scale's 
characteristics at the indi- 
vidual level are of some 
interest. Containing five 
items? the scale has a reli- 
ability coefficient of .^7. 
As earlier research 
(National Institute of Edu- 
cation? 1978) showed? boys 
are victimized more often 
than are girls. \/ictimiza- 
tion is correlated .24 with 
self-reported delinquency? 
implying a moderate tendency 
for persons who are victim- 
ized to engage more readily 
\r\ delinquent behavior them- 
selves. It correlates -.27 
with school attachment and 
-.28 with self-esteem. 
Interestingly? its highest 
correlate among the varia- 
Dies examined is gunishment 
(.35): Students who report 
more frequent personal vic- 
timization a^so more often 
report oeing punished in 
school ♦ 

iQ^aliditi^. "^here is 
al ways^some concern that 
students may not faithfully 
complete their question- 
naires? that they may fool 
around or give silly 
answers. As a check on 
this? a scale was included 
to detect unusual or nonsen- 
sical responses. This 
five-item scale is composed 
of items that a careful res- 
pondent would answer in only 
one way. It is kayed so 
that a rare response earns a 
point. This scale has not 
yet been used? but will be 
used later as a check on the 
results and as a quality 
control mechan ism. 
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The iten content of all 
student-level scales is 
shown in Appendix Table A* 

Measu r e s _ of Teachgrs 

The second largest group 
of inhabitants of a school 
environment are the teachers 
who work there* Students in 
the aggregate help to create 
an environment for the 
teachers f just as teachers 
create an env i ronment for 
the students. A characteri- 
zation of the teachers is 
important in describing a 
school or a project* 

Several of the action 
projects* theories lead to 
interventions geared toward 
teachers* The interventions 
are intended to improve 
classroom management? to 
change teachers' attitudes? 
or to involve them in new 
kinds of activities* One 
aspect of the evaluation 
therefore involves the meas- 
urement of teacher charac- 
teristics. Before turning 
to our account of school 
environmental measures? we 
will describe a set of indi- 
vi dur3l --1 evel teacher meas-- 
u r e s • 

f £2ziQi§9I§ t iorj_ A t ti t uge * 
This four-item scale is a 
measure of attitudes toward 
integrated education* It is 
incl uded because these 
del i nquency and school 
improvement programs are 
designed to provide services 
to heterogeneous groups of 
students * One component of 
several projects is training 
teachers to manage heteroge- 
neous classrooms and to 
interact with a variety of 
kinds of students* This 
scale is expected to be 



useful in ass es sing the 
effectiveness of teacher 
participation in such activ- 
ities* It has a reliability 
coefficient of ^69 (Table 9) 
and is relatively i n depen- 
dent of the other te.=*cher 
scale (Tables 10). As might 
be expected? nonwhites tend 
to score somewhat h i gher 
than whites on this scale* 

Job^Sati^sf ac t i^on. Thi s 
scale is composed of three 
of the four items in Hop- 
pock's (1935) scale of the 
same names> which has been 
used widely in research and 
has demon strated impressive 
ev i dence of conver gent val - 
idity (Robinson? Athanasiou? 
L Head? 1969) • Even short- 
ened to three items it has a 
reliability of .BO. It may 
conf i dentl y be taken as a 
measure of how well teachers 
I i ke the i r jobs • 

^SQii* "^^^is six-item scale 
measures the extent of out- 
of-class interaction that a 
teacher has with studentso 
Items ask about tutoring 
individual students before 
or after school and discuss- 
ing thei r per sonal problems 
with them. It fias a rel i 
ability coefficient of .67? 
and c orrel ates pos i t i vel y 
with Job Satisfaction? nega- 
tively with reports of 
cl ass room disruption? and 
positively with the extent 
of recent continuing educa- 
t i on act i v i t i es • 

I)tE§_^«.Sanct i_ons • Thi s 
is one of two scales devel- 
oped in an attempt to 
describe the types of 
responses to student conduct 
used by the classroom 
teacher* We are un aware o f 
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any short questionnaire 
measures of this aspect of 
classroom managements but 
provocative evidence from 
earlier research (McPartland 
C McOillf 1977; Gottfredson 
t Daigert 1979) suggests 
that responses to conduct 
are important in preventing 
disruption* Therefore we 
used the Dest advice we 
could get to develop lists 
of V ar ious ways classroom 
teachers might respond to 
student behavior. These 
lists became items in the 
questionnaire. Through fac- 
tor analytic examination and 
internal consistency item 
analysisf two scales 
emerged. 

The first set of items is 
termed ••Type A" Sanctions. 
A teacher who reports lower- 
ing grades as a punishmentt 
sending misbehaving students 
out of classf and paddling 
or reprimanding the students 
in class is given a high 
score. The scale has a 
rel labil ity of «.^7. Its 
largest correlate among tne 
other teacher measures is 
the amount of disruption the 
teacher reports; it is also 
moderately negatively corre- 
lated with nonauthoritar ian 
atti tudes. 

scale was^developed in the 
same way. In contrast to 
the Type A scalet which 
seems to include responses 
rooted in frustrationt Type 
B Sanctions a^eear to 
involve a wider range of 
resources. To earn a high 
score on this scalet a 
teacher reports giving extra 
schoolworkf awarding special 
privileges for good behav- 
iorf taking away privileges 
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for misconductf calling 
parentSf and referring stu- 
dents to the counselor or 
elsewhere. This five-item 
scale has a reliaoility of 
.60. It correlates only .16 
with Type A Sanctionst even 
though Doth scales would be 
elevated if a teacher fre- 
quently had to make some 
kind of response to miscon- 
duct. Jseful information 
about the construct validity 
of the t:wo sanctions scales 
can be obtained by examining 
their correlations with 
responses to a qjestion 
about home-based reinforcers 
(Table !!)• The use of 
home-based reinforcers to 
extend the range of rewards 
and punishments in the 
school appears to be a 
highly effective strategy 
(Barthf 1979; Atkeson t 
Forehandf 1979). Scores on 
the Type B scale correlate 
.35 with responses to this 
item% whereas those on Type 
A correlate only .07 (n.s.). 

Victi mization. As one 
way to Pleasure the amount of 
"de 1 i nquent behavior in a 
school f teachers are asked 
about their experiences of 
personal v i ct i mi zat i on . In 
the agqregatet these reports 
may be taken as an indicator 
of the amount of disruption 
in the school. An eight- 
item scalet asking about 
events ranging from obscene 
remarks or gestures to phys- 
ical attackf has a reliabil- 
ity of .67. 

Q.i^SL22!!!-iii5.Cii2ti2Q • ^ 
second way to assess the 
level of student misbehavior 
experienced by a teacher is 
provided by a two-item 
classroom disruption scale. 
It asks to what degree 
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classroom disruption inter- 
feres with teaching? and how 
much of the teacher's time 
is devoted to coping with 
disruptive students* Its 
rel i abi 1 i ty is .TO^ 

L ow_E xg ec t a tj. on s • A 
labeling theory perspective 
implies that teacher expec- 
tations for student perfor- 
mance may become incorpo- 
rated .into the student's 
Self-concept and result in 
misconduct or poor academic 
performance* To provide a 
measure of this variable? a 
two- i tern Low Expectat ions 
Scale asks teachers to judge 
what percentage of their 
students are of low ability 
and have "behavior prob- 
lems."' The scale has a 
reliability of •57 • It cor- 
relates -#24 with Job Satis- 
faction and •^3 with Class- 
room Disruption* 

QggOQgg s to^^ Student Sug- 
gest io ns* This two- item 
scale has very low reliabil- 
ity* It askSf for example? 
how often teachers change 
their lesson plans to acco- 
modate student suggestions* 
It was intended to provide 
an index of teacher respon- 
siveness in order to assess 
the effectiveness of train- 
ing programs or other inter- 
vent ions with teachers* 

£l2f § s s iona i_D(6yeXo£men t * 
Eight items form a scale 
measuring the extens i veness 
of recent cont i nu ing educa- 
tion or in-service 1 earning* 
This scale? with reliability 
*7A? is for use in document- 
ing the implementation of 
training components* It 
also helps to lend evidence 
of construct validity to 
other teacher measures* The 



correlations in T^ble 10 
general 1y accord with the 
interpretation that teachers 
scoring high in pr ofessional 
devel opment a re more satis- 
fied? interact more with 
students? and ara more open 
to student suggest i ons* 

tti^Sl* Intended in part to 
measure sympathetic atti- 
tudes (as one way to get at 
the "car i ng? competent 
taacher" constellation)? a 
measure of puni ti ve moral i sm 
is included* To earn a high 
score on this scale? a 
teacher rejects such items 
as? "A few pupils are just 
young hoodlums and shoul d be 
treat ed accordingly." This 
three-item scale has a reli- 
ability of m5^» 

The item content of all 
taacher-level scales is 
shown in Appendix Table B* 

tI§§sures_of _School^_C2ima te 

As discussed earlier? the 
assessment of school cli- 
mates is fundamental 1 y di f- 
ferent from the measurement 
of individuals* Whereas 
individual differences are 
the entire point of measure-- 
ment at the individual 
level? these differences are 
"error" or "noise" in the 
assessment of an environment 
based on the reports of its 
i nd i V idua 1 inhabi t ants* 
Consequentl y? environments 
are someti mes character i zed 

aggregated or averaged 
reports of individual s* We 
have constructed cl imate 
scales based on such aggre- 
gated reports? and someti mes 
describe cl i mates using 
averaged characteristics of 
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individuals (cf. Astin L 
Hollands 1961) • For climate 
scalesf reports are first 
averagedf and then item 
analyses proceed based on 
school means for the items. 

Measijres^of^Cliisate^Bas^^ 
.Student^Rgegrt s 

Conmurjit jt^Criise. Th i s is 
a three- item^scal e based on 
averaged responses to ques- 
tions about whether there 
are gangs in the student's 
nei ghborhoodf whether the 
gangs try to get the student 
to join and whether the stu- 
dent's parents were robbed 
in the last year# This 
scale may be useful in 
describing the community 
context of the school (cf. 
National Institute of Educa- 
tiont 1978). It has a reli- 
abil ity of .57. (An «out- 
liert" i.e.f a school with 
extreme values on many 
items^ was dropped from the 
sample in these and subse- 
quent item analyses reported 
heret possibly deflating 
most reliabilities somew- 
hat.) 

G angs _in_Sc hoo 1 . This 
scale is^composed of aver- 
aged responses to questions 
about whether there are 
gangs in the school andt if 
sot how much trouble they 
cause. The reliability of 
this scale is .80. 

Saf ety. Thi s is a 
l3-item scale asking if stu- 
dents stay away from any of 
a 1 ist of places in the 
school. It also asks if 
students feel safe at 
schoolt or if they fear 
someone will hurt them at 
school or on the way to 
school. It resembles what 



was called "School Climate" 
in the Schools Initiative 
Evaluation (Grant et al.f 
1979). Its rel iability 
coef f ic ient i s .92. 

OisruBtion. This four- 
item~cale"Ts based on aver- 
aged responses to questions 
about the students" having 
to fight to protect them- 
selvesf seeing' teachers 
threatened or attackedt and 
being in classes that were 
totally stopped by disrup- 
tive students. It has a 
rel iabil ity of .^2» 

Indi vidualized^Instruc- 
tionT "bis scale is an 
attempt to use student 
reports as evidence about 
the level of individualized 
instruction characterizing 
tne school as a whole. 
TndTviduaTized instructiont 
as usually construedt 
involves the development of 
individual learning planst 
rewards for improvement over 
past levels of performancet 
and a pace of instruction 
suited to the individual. 
Two aspects of this concep- 
tion are incorporated in 
this medsure--students' 
reports that they have indi- 
vidual learning planst and 
reports that they can work 
at their own speed in class. 
The reliability coefficient 
is .58. 

Student^Disr es2S£l* 
theoretical ""per soective 

(Greenbergt 1977) assumes 
that delinquency is in part 
a result of a special status 
accorded youtht one which 
isolates them from meaning- 
ful adult roles and subjects 
them to degrading interper- 
sonal exchanges to which 
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adults would not be 17— 

subjected* This scale is 
intended to assess the 
degree to which students 
feel that a school environ- 
ment as a whole either 
degrades them or treats them 
with dignity* A low score 
could indicate that students 
feel they are treated with 
dignity* The items aret 
••Students are treated like 
children here;'' '•Teachers 
treat students with 
respect;" and "Teachers do 
things to make students feel 
put down*" Its reliability 
coefficient is *7e* 

Student^TeacJier^I^nter ac- 
tion* This scale aims to 
assess the degree of out-of- 
class positive social inter- 
action with teachersf from 
the students* point of view* 
It is based 00 the averaged 
responses to two items: "I 
talk to some of my teachers 
about things other than 
school work; •• andt "Teachers 
help me with schoolwork out- 
side of class*" Its reli- 
ability is *60* 

£l5DQiQ3_3nd_ Action* 
This scale is Intended to 
assessf from the point of 
view of the studentst the 
degree to which schools 
engage in experimenting and 
probl em- sol ving? or the 
degree to which they resist 
change* It is composed of 
the following three aggre- 
gated items: "it is hard to 
change the way things are 
done in this school"; "The 
teacher s and principal in 
this school make plans to 
solve problems"; and "This 
school hardly ever tries 
anything new*" It has a 
reliability coefficient of 
*65* 



Fa i r ness * Evi dence i s 
accumulating that the degree 
to which students perceive a 
school^s rules as fair and 
clear is associated with the 
degree of orderl iness of the 
school (Nat i onal Institute 
of tducdtiont 1978; Gott- 
fredson G Daigert 1979)* 
Consequentl yt seal es 
designed to assess these 
constructs were devel oped* 
Fairness is a three-item 
agqregate-1 evel scale based 
on student reports thot the 
rules are fair? that the 
pun i shment for break i ng 
rules is the same for every- 
one t an J that the pr incipal 
is fair* It has a reliabil- 
ity of mbZm 

Cl^ari^tji* Intended to 
measure the clarity of 
school rules from the point 
of view of the school •s stu- 
dents? this scale is com- 
posed of questions asking 
whether everyone knows what 
the rules aret whether 
teachers let the students 
know what is expected? 
whether the pr i nc i pal is 
f i rnu This f our- i tern scale 
has a reliability coeffi- 
c i ent of #5^* 

Student^ In f l^uence* It is 
often assumed that^student 
influence on the way a 
school i s run may 1 ead to a 
number of positive outcomes* 
This six-item scale is 
intended to assess how much 
i nf 1 uence students have in 
their schools* Sampl e items 
i nc 1 ude: "Students have 
little say in how the school 
is run"; "Students have 
helped to make the school 
rules"; and "Students are 
seldom asked to help solve a 
problem the school is hav- 
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ing." The scale's 
reliability is •62» 

G roupi ng^ This scale 
assesses the students* per- 
ceptions of groupingt or 
segregation of students with 
spec iai characteristics 
within the school. It is 
composed of the following 
three items: ^students of 
different races usually end 
up in different classes"; 
andf "This school has spe- 
cial classes for slow lear- 
ners**; andt "There are spe- 
cial classes for trouble 
makers." Its reliability is 
• 55* 

The correlations among 
the scalesf shown in Table 
13f are not low enough to 
imply that each scale meas- 
ures an important indegen- 
dent dimension of school 
climate* In particulart 
scales 6 through 11 show 
considerable redundancy? 
implying that they should 
probably be interpreted as a 
group* Small differences in 
elevation among these scales 
should be interpreted with 
caution* The item content 
of these scales is summa- 
rized in Appendix Table C* 

It seems appropriate to 
reduce this set of a priori 
scales through combination* 
This could result in produc- 
ing the information with 
fewert longert and more 
reliable measures* 

Clii5ate_Scales_Based_on 
iiacher^^Refiorts 

An alternative perspec- 
tive on the climate of a 
school is provided by the 
reports of teachers * 
Accordingly f seven climate 



scales were constructed from 
the teacher quest i onnai re t 
using averaged tejcher 
responses about their 
school* The item content of 
these seven scales is pre- 
sented in Appendix Table 0* 
Their reliabilities are 
shown in Tabl e 15* 

^§sources_f or _Inst ruc- 
tion! hls^'scale is 
Intended to measure relative 
levels of resources (equip- 
mentf materials? learning 
opportunities) available in 
the school* It contains 
items asking about teaching 
suppl iesf spacet extra- 
school settings used for 
instructiont and timeliness 
of avail abil i ty of 
resources* This four-item 
scale has a reliability of 
*86* 

I nvolvemen t_of _2.2££!l5.i 
and^^Communrix* A goal of 
the^Al tirnati ve Education 
Program is to increase the 
use of community and family 
resources by schools as a 
structural school improve- 
ment* This scale seeks to 
assess parent and community 
involvement according to 
aggregate teacher reports* 
It asks about parent influ- 
ence on policies or prac- 
ticesf direct parent assist- 
ancet relations between 
parents and teacherst and 
community recept i veness* 
The six-item scale has a 
ral iabil i ty of *80* 

menti ~Some evidence sug- 
gests that the commitment of 
an organization's staff is 
related to project implemen- 
tation (Grant et al*f 1979; 
Barman & r^cLaughlint 1976)* 
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Accordinglyt a two-item 
scale to assess staff com- 
mitment was included* Its 
rel labi 1 ity is •SZ* 

Staf f _MgraJ[e# As wi th 
commitment f morale is some- 
times suggested as a conco- 
mitant of success in imple- 
menting innovat ionSf and i t 
i s an i mpor tan t character i s- 
tic of an organization in 
its own right* An ll-item 
scale containing items such 
asf "Our problems in this 
school are so big that it is 
unrealistic to expect teach- 
ers to make much of a dent 
in them;** and "(Is the 
teaching faculty) frus- 
trated?" Its reliability is 
• 90* 

Planning and A c t i on • 
Presumabl yt or gani zat ions 
engaging in systematic plan- 
ning and that are open to 
change are most likely to 
successfully implement inno- 
vations* Based on this 
assumptiont we constructed a 
nine-item scale to assess 
planning and action* It 
askSf "HOW often do you work 
on a planning committee with 
other teachers?" "(Is the 
principal ) progressive?" 
"(Is the teaching faculty) 
open to change?" Its re1i- 
abil ity is *87* 

il222tl3-i^!DlQis t r a tion * 
Our earlier research (Gott- 
fredson t Oaigert 1979) sug- 
gests that the way a school 
is run is important in 
understanding ita climate 
and i n prevent i ng school 
disruption* To the best of 
our knowledge* detailed stu- 
dies of school admi ni str a- 
tion tend to focus on the 
personal characteristics of 
administrators (e*g*f Minert 



1967) f or are ethnographic 
or ofaservat ional accounts of 
the typi cal act i vi ti es of 
administrators* Here we 
wished to assess the percep- 
tions of administrative 
style and procedures from 
the point of view of the 
body of teachers who experi- 
ence them* Accordingly? we 
constructed a 12-item scale* 
T y p i cal i tems are: " Sim:) let 
non-t i me cons urn i ng p roce- 
dures exist for the acquisi- 
tion and use of resources;" 
"There is little teacher-ad- 
ministration tensi on in this 
school*" "(The principal 
is) open*" In a sense this 
scale represents a global 
rating of the positiveness 
with which teachers view the 
s c h oo 1 s^ s administration?^ 
although the item content 
focuses on both pr i nc ipal 
behavior and some probable 
practical consequences of 
t ha t behav i or* Its reli- 
ab i 1 i ty is *9Z* 

tion_and_Gr ad^ng* T he 
Al ternati ve E ducat ion Pro- 
gran seeks to create struc- 
tural changes in schools to 
i ncrease ind i vi dual i zed 
ins tr uct iont and this inter- 
vention i s pi anned by sev- 
eral of the action projects* 
Accor dingl y f th i s f our- i t em 
scale aims to measure indi- 
vidualized instruct! on by 
asking if i nd i v i dua 1 i zed 
learning plans are usedt and 
if grading is based on 
i mp rovement ver s js "the 
curve*" 

Schogl_Rac e_Re 1^3 1 ions * 
This br i e f two- i tern measu r e 
asks about race relations 
from the teacher point of 
view* It asks how well dif- 
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ferent groups get along. 
Its rel iabil i ty is •llm 

I nt er ac t ion_wXth_S t u- 
dents* The Alternative Edu- 
cation Program assumes that 
••caring competent teachers** 
Mill foster prosocial out- 
comes and prevent delin- 
quencyt and several action 
projects aim to alter teach- 
er-student rel at ions • 
Interpreting what ••caring 
and competent** means is dif- 
ficultf but as one way to 
get at this constellation we 
created an Interaction with 
Students Scale. This six- 
item index asks about the 
frequency of teacher inter- 
action with students and 
about how well students and 
teachers get along. Its 
reliability is .ao. 

iQi®aL3tion_vs«_Searei^* 
tion^bi AbilU^^or^Conduct. 
This scaTe'^Ts also included 
to measure an aspect of 
project impl ementati on 
sought by the Alternative 
Education Program: the 
avoidance of tracking or 
isolation. The six-item 
seal e contains items such 
as: •'Students of mixed 
ability work together in 
small groups in my class;** 
••This school has special 
classes for slow learners;** 
and "in this school there 
are special classes for stu- 
dents who repeatedly misbe- 
have.- Its reliability is 
•55f and the appropriate 
interpretation of the scale 
is unclear. Opinions differ 
about the wisdom of homoge- 
neous vs. heterogeneous 
grouping according to stu- 
dent conduct or academic 
performancet although the 
current climate^ and some 
evidence (Slavint I980)f 
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iJiplies that heterogeneous 
grouping can have some vir- 
tue. 

Student Influence. Stu- 
dent participation in school 
decision making is one of 
the major structural ele- 
ments the Alternative Educa- 
tion Pragram wants to create 
through the action projects. 
The assumption aoparently is 
that student influence will 
help to create other benefi- 
cial structural changest or 
it may contribute to 
decreased alienation or 
sense of powerl essness. 
Measures of student influ- 
ence used in previous stu- 
dies (National Institute of 
Education* 1978; Gottfredson 
L Daigerf 1979) assessed a 
limited range of influence* 
and certainly do not assess 
the kinds of student influ- 
ence possible. Therefore* 
although based on the scale 
used earlier by Sottfredson 
L Oaiger (1979)* this scale 
is expanded somewhat (to 
five items). Sample ques- 
tions are ••! often change my 
lesson plans based on stu- 
dent suggestions;" and 
••Teachers and their students 
^ork together to make rules 
governing behavior in the 
classroom.'* The scale has a 
reliability coefficient of 
.81. 

Pr of essional_Oeyeloament. 
Tnis'^scaTi Is the aggregate- 
level counterpart of the 
i ndi vidual-1 evel teacher 
Professional Development 
Scale. Because of the way 
it is constructed* it i s of 
use only for characterizing 
the 1 evel of professional 
development activities for 
the school as a whole. The 
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eight-item scale has a reli- 
abil ity of •86« 

or Lack of Safety^ Intended 
to be one measure of delin- 
quent or disruptive behavior 
in school? this 12-item 
scale asks about time spent 
coping with disruptive 
behavior and about percep* 
t ions of saf ety* It is 
highly reliable (•93) 

Success_OoQortuni t i es • 
Many critics of tradition- 
ally structured schools 
(e«g* f Howard f 1978) argue 
that schools are "rigged** 
aga i nst 1 ow ach i ev i ng stu- 
dents: Many students exper- 
ience only failure in 
school* This scale is an 
attempt to collect teacher 
opinions about the e'xtent to 
which their schools are 
••unrigged" by providing suc- 
cess opportunities* It asks 
whether any students can 
earn high marks? and whether 
students can get special 
privileges for their perfor- 
mancG* We have some douDts 
about the meaning of this 
scale: , It is heterogeneous 
in content as well as brief? 
and it has modest reliabil- 
ity (•60)* But it has some 
interest and may be of some 
ut i 1 ity* 

t i on • The use of grades as 
a response to mi sconduct i s 
correlated with school dis- 
ruption rates (Gottfredson L 
Oaigerf 1979). On the face 
of itf this also appears to 
be a poor practice because 
it makes the grading and 
sanctioning process ambigu- 
ous* A two- item index uses 
teacher reports to charac- 
terize the extent of this 



practice in schools* It has 
a reliability of *84* 

Tnt ergreting^Scores^f or 
Sch oo l s 

One way of interpreting 
scores on all these scales 
is by using a profile sheet 
that enables the examination 
of the standing of a school 
relative to a "norm group*" 
Profile sheets have been 
prepared that do just this? 
using the school s in the 
School Act i on Effectiveness 
Study as the norm gr oup *<2> 
Remember that this is not a 
rep re sent at i ve sampl e of 
schools* In general ? these 
schools are included in the 
sample at least in part 
because they indicated in 
thei r appi i ca t i ons that 
cheir problems of crime? 
dropout? and nonat tendance 
are relatively severe* 
Never thel ess ? the group does 
provide some basis for com- 
parison* 

Profile sheets are used 
to plot t-scores* This 
means that the "average" 
school woul d have a score of 
53? and that the standard 
deviation i s 10* As an aid 
to interpretation? si^ bar on 
the profiles shows the 
range of the middle 50% of 
the scores for each scale* 
This is especially useful 
when distributions of scores 
are skewed? or when they are 
flatter or more peaked than 
i n a normal distribution? 
because the bar acts as an 
easy guide in interpretiwj 
the school's score* 

For many of these meas- 
ures? scores are not very 
reliable* Remember to take 
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reliability into account 
when making interpretations. 
In additiont "rel iabi 1 i ty** 
applies to scores in gen- 
eral t not the score for any 
particular school. When 
only a small number of per- 
sons reported about a 
school • s cl imatet a score 
may contain lots of error. 
Similarly^ if response rates 
were lowt the score may con- 
tain bias. This is because 
students who completed the 
questionnaire may not accu- 
rate! y represent the 
schools' students in gen- 
eral. Oo not make much of 
small differences in scores 
in these profilest and be 
skeptical about profiles 
based on small samples. An 
interim rule of thumb to 
follow for profiles based on 
50 or more questionnaires is 
to ignore the differences of 
five t-score units or less. 
When fewer than 50 question- 
naires contribute to a 
school's scoret even larger 
differences should be 
ignored. 

If several sources of 
information converge in sug- 
gesting the same interpreta- 
tions the results are worthy 
of careful consideration. 
But remembert a survey like 
this is only one source of 
information about these 
schools. No survey or set 
of scales can provide a 
magic picture of an organi- 
zation's envi ronment . 

Th e y ses_of _Scor es_f 0£_lndi* 
" "v i dual s 

All individual-level 
scores are confidential and 
are used for research pur- 
poses only. (In future 



appl icat i onsf similar scales 
could be used in counseling 
or""for diagnostic purposes. 
They can not be used in 
those ways in this Program 
because of the assurances of 
confidentiality given to 
those who completed the 
instruments on which scores 
are based.) Elsewhere in 
this reportf and in subse- 
quent reports to be issued 
by the SAtSt these scores 
are used in detailed exami- 
nations of the effects of 
project components • When 
individual-level scores are 
aggregated to the school 
level f they can be used to 
.ctescribe schools in terms of 
the characteristics of their 
studentries. In profiles we 
make available to project 
directorsf these aggregated 
scores are also presented as 
t-scoresf where 50 is the 
mean aggregated score for 
schoolSf and 10 is the 
school^standard deviation. 

The_ytilitY_of_lQf ormation 
?or]]Pro ject^Managers 

In workshops conducted in 
Augustf I981f school pro- 
files were made available to 
project directors. These 
profiles provide assessments 
of schools useful for diag- 
nostic and prescriptive pur- 
poses. The efforts of thou- 
sands of students and 
teachers in completing these 
surveys will go partly to 
waste if this information is 
not used in project plan- 
ning. 'Ae earnestly hoped 
that this information would 
be usedf and are gratified 
that several projects have 
made extensive use of this 
information in renewed proj- 
ect planning. 
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Similarlyt interim feed- 
back Me have provided to 
project directors on the 
characteristics of th^ir 
cl ientel e (in summary form) t 
and about the effectiveness 
of their interventions based 
on the statistical analyses 
of individual scales is 



intended to be used in 
refining interventions* No 
one expects to see dramatic 
effects of projects in thei r 
davel opmental s tages • Proj- 
ects will increase in effec- 
tiveness largely by using 
the information provided by 
this interim feedback* 
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Footnotes 



1. For more thorough discussion see Throndike (1971). 

2. One school with extremely deviant scores on a number 
scales was excluded from the norm group. 
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Figure 1 . The interpretation of standard scores 
in a normal distribution 
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Table 1 

Selected Characteristics of National Samples of Students 
and the School Action Effectiveness Study Sample 



Ethnic 

Identification 



Elementary 
& Secondary 
Enrollment 



High 
School ^ 
Seniors 



SAES ^ 
Sample 



American Indian 

Asian- American 

Spanish- American 

Black 

White 

Other 



0.8 
1.2 
6. A 
15.5 
76.0 



1 
0 
3 
11 
80 



2.9 



2.0 
1.1 
25.4 
43.7 
25.6 
2.1 



Father 
Present? 



Americans 
Aged 18 and 
Under 



High 
School ^ 
Seniors 



SAES 



Yes (%) 



79.0 



81.7 



56.8 



Type of Personal 

Victimization : 

Physical attack 
Robberies of more than $1 
Personal theft of nore than $1 



P e rcenta^e Reporting in Mont h 
-'^ SAES 



National Sample^ 



9.8 
3.4 
27.6 



10.3 
6.5 
23.0 



Self-reported 

Delinquent 

Behavior 



Percentage Reporting at Least One, Past Year 



Destroy or damage 
school property 

Stole something worth 
more than $50 

Carried a hidden 
weapon 

Gang fights 

Hit teacher 

Hit students 

Joy riding 

Break in 



National 
Sample^ 


SAES^ 


11.4 


11.1 


2.6 


6.7 


6.3 


12.0 


7.9 


10.4 


6.6 


9.7 


5.0 


45.4 


A. 5 


6.5 


2.5 


5.1 



a . 
b. 
c . 
d. 
e . 
f . 



Fall 1976 OCR data 
Monitoring the Future, 1976 
Unweighted 
CPR 

Monitoring the Future, 1976 

Safe School Study 

National Youth Survey, 1978 
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Table 2 



Scale 



Family background ^ 
Parental education 
Parental emphasis on 
Maternal role model 
Maternal role model 
Paternal role model 
Paternal role model 



education 
(negative) 
(positive) 
(negative) 
(posi tive) 



Social Relations 

Attachment to parents 

Negative peer influence 
Parental supervision 

Attitudes and Social Development 
Alienation 

Attachment to school 

Belief in rules 

Interpersonal competency 

Involvement 

Positive self-concept 

Practical knowledge 

Rebellious autonomy 



R.U.blUcy coefficients (Alpha) for Individual-Level Studenc Scales 

by Condor and -ELl.nlc Solf-IdentlCicaLxon ^^^^.^^^ 

u . Blacks Whites , _Lnd'j!Jl_ 
^TT^w.^TTmI^^MmI Male Fen ul o M>U e Fcmale _il: >^^ '^"-^'^ "-^^ -^ 



Asian 
/\ii}erican 

Total Number 
Male Female Satr.ple of items 



^7 
39 

67 



^0 
75 
U(x 
Ul 
63 
56 
67 
^5 



39 

57 
6d 



39 
70 
U'l 
A3 

51 
66 

52 



73 

A7 
20 
At) 
56 
7A 



72 
39 
A2 
AA 
57 
72 



37 
7A 
53 
AO 
6A 
60 
70 
A2 



45 
70 
A 2 
A5 
6A 
5? 
70 
59 



A2 
35 

i7 
53 
72 



35 
72 
51 
A6 
63 
52 
71 
31 



38 

33 
A5 
51 
69 



52 


66 


50 


61 


52 


59 


65 


60 


61 


56 


55 


57 


27 


23 


27 


lA 


11 


17 



AA 
71 
A3 
36 
65 
6A 
67 
A5 



77 
A2 
A6 
53 
53 
77 



50 
60 
26 



33 
67 
AA 
AA 
66 
59 
71 
3A 



78 
Al 
35 
A3 
58 
71 



39 
67 
A2 
30 
62 
61 
72 
A2 



A8 
10 
55 
56 
70 



56 
81 
59 
AA 
61 
73 
77 
A5 



56 
31 
51 
A8 
75 



53 
81 
52 
37 
59 
69 
76 
53 



8A 
A3 
09 
A9 
5A 
50 



A6 
79 
5A 
A 6 
62 
70 
7A 
Al 



72 
AA 
25 
A8 
62 
75 



60 


72 


81 


66 


78 


53 


71 


73 


72 


69 


17 


31 


29 


31 


AO 



56 
79 
A9 
3A 
62 
69 
7A 
A9 



86 
A5 
09 
32 
AS 
55 



39 
79 
70 
AO 
59 
71 
69 
59 



68 
62 
A7 
62 
'71 
75 



AA 
75 
56 
AA 
63 
67 
76 
60 



66 
50 
61 
30 
75 
57 



06 
81 
51 
Al 
80 
65 
63 
A2 



62 
55 
53 
58 
22 
73 



55 
72 
67 
32 
52 
78 
5A 
AO 



76 
A6 
36 
A6 
55 
71 



68 


77 


73 


73 


61 


60 


62 


7A 


60 


6A 


28 


b 


15 


5A 


27 



A A 
75 
50 
A2 
6A 
63 
71 
A6 



A 

10 
6 
5 
15 
12 
7 
3 



Behavior 

School effort 
School nonattendance 
Self-reported delinquency (total) • 
Self-reported drug use 
Self-reported serious delinquency 

School Experiences 
School punishments 
School rewards 
Victimization 

Validity 
Invalidity 



39 
63 
31 
71 
72 



AA 
6A 
79 
71. 
7A 



37 
6A 
87 
72 
80 



37 
61 
77 
67 
61 



53 
57 
81 
72 
75 



51 


58 


59 


58 


49 


50 


53 


57 


56 


55 


71 


67 


72 


64 


68 


A5 ■ 


36 


AO 


AO 


51 



AS 
61 
71 
66 
59 



50 
53 
65 



Al 



51 
65 
85 
73 
80 



50 
59 
76 
65 
69 



67 
7A 



79 
85 



65 
69 
8A 
79 
79 



62 
69 
87 
73 
82 



63 
71 
83 
80 
73 



A8 


AA 


57 


5A 


51 


A6 


5A 


50 


52 


5A 


51 


47 


72 


56 


63 


61 


70 


61 


52 


A6 


A8 


33 


50 


36 



52 
62 
85 
78 
74 



Note; Decimals are omitted. 

•K.U..iUclos osCi™.C.d on entire .»ple. no sep.r.te c.lcul.rlons wore ,„ade for construction and hold-out samples. 
^Reliability too low to estimate. 

*Too few cases with complete data to estimate reliability, 



47 
60 
81 
80 
76 



62 
74 
93 
53 
92 



65 
29 
63 
71 



51 

66 
84 
75 
79 



47 


32 


64 


70 


54 


49 


64 


66 


52 


54 


6,6 


48 


82 


69 


68 


53l 


02 


55 


57 


47 



5 
2 
19 



fasilly bickground 
Parental education 
Parental emphASls on education 

Maternal role model (negative) 

Mattrnal role modal (positive) 

Paternal role modal (nagatlve) 

Patarnal rol« model (positive) 

Social relations 

Atrachnent to parenta 
NeRAtiva p««r influenca 
Parental suptrvlslon 

Attlcudet and toclal development 
Alienation 
Attachiient to tchool 
Belief in rulet (-) 
Interpersonal competency 
Involvement 

•Positive self-concept 
Practical knowledge 
Rebellious autonoTsy 



Behavior 

School effort 
School non-attendance ' 
SelNreporcbd delinquency (total) 
S«lf«reported drug uaa 
Self-reported serious delinquency 

School sxpariences 
School punishments 
School rewards 
Vlctlnixaclon 

Validity 
Invalidity 



Tabic 3 

Means and Standard Deviations of Raw Score Individual-Level Student Scales 



Spanlsh-Amerlcan 



Black 



White 



Hale 



A gtcrican Indian 

Male 



Tenia Ic 



M.ile 



MSD 



SD 



SD 



SD 



SI) N 



SD 



3.20 


2 


57 


536 


2 


.56 


2 


30 


616 


4 


.84 


2.i5 




4 




-.36 


2 


52 


439 




.29 


2. 


41 


456 




.25 


2.25 


m 






.31 




57 


472 




.35 




68 


509 




.57 




.87 


342 




55 




1 


13 


442 


• 4 


.78 


1. 


13 


469 


4 


.88 


1.20 


332 


4 


93 


.77 


1 


05 


441 




.84 


1. 


10 


461 




.78 


1 


.04 


308 


1^03 


4.88 


1 


53 


437 


4 


.54, 


1. 


58 


442 


4.85 


1 


.60 


3? 4 


4 


56 


.85 


3 


04 


399 




.47 


3.37 


442 




.18 


3 


.16 


567 




22 




4 


63 


524 


-1 


.17 


3. 


72 


581 




.39 


4 


.62 


850- 


1 


10 


1.51 




65 


464 


1 


.70 




53 


523 


1 


.25 




.73 


669 


1 


55 


1.37 


1 


11 


445 


1 


.25 


1. 


12 


520 


1 


.35 


1 


.08 


719 


1 


31 


-.46 


5 


60 


427 


1 


.17 


4. 


90 


514 




.37 


4 


.95 


692 


1 


40 


2.14 


1 


53 


415 


1 


.67 


1. 


31 


492 


2 


.36 


1 


.42 


652 


1 


93 


3.70 


1 


18 


427 


3 


.78 


1. 


18 


506 


3.86 


1 


.14 


678 


4 


05 


'1.61 


5 


65 


496 


-1 


.38 


5. 


74 


533 




.06 


6 


.17 


761 




77 


-1.24 


5 


37 


281 




.06 


4. 


98 


342 




.39 


5 


.03 


426 


2 


17 


11.91 


2 


94 


451 


11 


.58 


3. 


13 


524 n 


.95 


2 


.95 


769 n 


80 


1.76 


1 


00 


446 


1 


.66 


1. 


10 


518 


1 


.84 




.95 


674 


1 


81 


7.04 


1 


98 


557 


7 


.63 


1. 


91 


585 


7 


.74 


1 


.87 


889 


8 


37 


1.73 


1 


96 


620 


1 


.43 


1. 


72 


636 


1 


.12 


1 


.67 


1081 




85 


2.28 


3 


20 


326 


1 


.18 


1. 


92 


394 


2 


.68 


3 


.21 


271 


2 


17 


.84 


1 


21 


356 




.65 


1. 


04 


432 




.93 


1 


.29 


489 




95 


.98 


1 


79 


345 




.33 




85 


424 


1 


.39 


2 


.09 


456 




49 


.19 


2 


77 


545 




.86 


2. 


14 


610 




77 


2 


.78 


865 




16 


..42 


3 


47 


498 




.58 


3. 


38 


570 




.48 


3 


.34 


764 




13 


1.04 


1 


49 


568 




.64 


1. 


10 


618 


1 


.22 


1 


.59 


891 




8i 


1.24 


1 


14 


470 




.94 


1. 


04 


532 


1 


.09 


1 


.19 


754 




77 



2.23 11?7 
2.20 365 



.79 
1.10 
1.19 
1.57 



3.25 
3.60 
.62 



1.10 
4.61 
1.33 

.96 
6.06 
4.42 
3.03 

.99 



408 
388 
361 
362 



702 
9G7 
853 



5.22 
-.59 

.66 
4.66 

.93 
.5.09 



-.34 
1.69 
1.30 



856 
820 
737 
815 



1.42 
•1.71 
2.21 
3.84 
849 -1.7G 
549 -'.22 
894 11.88 
8n 2.09 



1.78 996 
1.39 1121 



2.39 
1.18 
1.12 



2.35 
3.20 
1. 13 



295 
557 
525 



7.31 
1.50 
3.57 
1.45 
1.45 



985 .08 
906 -1.32 
992 1.00 



2.32 
2.61 
.75 
1.26 
1.12 
1.12 



3.49 
5.40 
.73 



1.18 
6.31 
1.53 
1.17 
5.25 
5.69 
3.08 
.93 



2.13 
2.03 
3.73 
1.46 
2.23 



2.53 
3.00 
1.42 



^^3 
138 

146 

148 

137 

138 



186 
467 
434 



435 
463 
325 
333 
403 
271 
507 
351 



426 
623 
178 
420 
206 



580 
402 
541 



5 


.18 


2. 


33 


430 


4 


30 


1. 


96 


53 


4 


40 


2. 


19 




5 


28 


2 


55 


21 


4 


56 




. f>S 


16 




32 


2 


54 


l62 


— 1 


6 2 


^ 


80 


4 2 




14 


^* 


60 


\l 




62 


2 


.88 


31 




75 


•J 


7 2 


20 




64 




79 


162 




6 1 




72 


5 1 




78 




9 8 


54 




59 


][ 


01 


32 




29 




.64 


21 


4 


76 


1 


22 


160 


3 


85 


1 


05 


48 


4 


96 


1. 


31 


49 


4 


87 


1 


09 


31 


3 


80 


1 


.28 


20 


1 


03 


1 


23 


160 


1 


22 


1 


1 5 


46 


1 


00 


1 . 


3 5 


L 1 
** J 




06 




44 


31 




68 




.84 


22 


4 


60 


1 


56 


159 


4 


85 


1 


23 


40 


4 


18 


1. 


77 


45 


4 


67 


1 


40 


30 


4 


45 


1 


.60 


22 




81 


4 


18 


230 




32 


■5 


91 


4 8 




77 




l\ 


50 




60 


4 


26 


36 


-1 


47 


4 


.44 


24 




01 


4 


63 


561 


2 


99 


5. 


13 


81 




19 


4. 


46 


84 




91 


5 


88 


43 




23 


4 


.27 


33 


1 


51 




67 


496 


1 


09 




76 


78 


1 


51 




60 


73 


1 


20 




.72 


51 


1 


42 




.76 


31 


1 


31 


1 


22 


572 


1 


63 


1 


17 


73 


1 


27 


1. 


13 


70 


1 


30 




95 


50 


1 


41 


1 


.23 


34 




28 


5 


84 


534 


-1 


89 


6 


40 


68 


1 


07 


5. 


04 


68 




23 


6 


19 


48 


I 


75 


4 


.70 


32 


1 


56 


1 


32 


396 


' 2 


48 


1 


82 


64 


2 


27 


1. 


52 


56 


2 


56 


1 


58 


41 


2 


03 


1 


.65 


30 


4 


08 


1 


00 


409 


3 


92 


1 


12 


59 


3 


f;2 


1. 


18 


61 


3 


78 


1 


15 


46 


3 


87 


1 


.02 


31 




48 


5 


76 


443 


-1 


19 


5 


61 


67 




79 


6. 


25 


76 


2 


15 


8 


33 


41 


2 


44 


5 


76 


29 




84 


4 


99 


330 




51 


5 


52 


37 




35 


5. 


32 


45 


-2 


04 


6 


10 


34 


2 


24 


5 


02 


24 


11 


89 


3 


10 


597 


11 


56 


3. 


13 


72 


]L87 


3. 


18 


75 


12 


42 


2 


46 


52 


12 


12 


2 


41 


34 


2 


18 




94 


409 


2 


.18 




98 


65 


2 


07 


1. 


03 


61 


1 


75 


1 


01 


44 


I 


79 




99 


33 


8 


16 


1 


93 


492 


7 


31 


1. 


98 


80 


8 


13 


1. 


83 


78 


7 


52 


2 


20 


42 


8 


58 


1 


87 


36 


1 


34 


1 


95 


700 


1 


64 


1. 


98 


104 


1 


71 


1. 


96 


99 


1 


54 


2 


IS 


63 


1 


07 


1 


49 


41 


2 


23 


2 


80 


218 


5 


02 


4. 


00 


44 


4 


72 


3. 


47 


43 


3 


93 


5 


14 


14 




90 


J 


10 


10 


1 


74 


1 


65 


491 


1 


98 


1 


66 


65 


2 


28* 


1 . 


75 


60 




91 


1 


08 


23 




50 




80 


12 




58 


1 


27 


248 


1 


94 


2 


20 


49 


1 


19 


1. 


81 


54 


2 


60 


3 


52 


15 














64 


2 


14 


669 




31 


2 


60 


91 




55 


2. 


03 


82 




39 


2 


96 


55 




31 


2 


75 


40 




39 


2 


95 


469 




04 


3 


24 


76 




61 


3. 


69 


70 




21 


3 


62 


46 




76 


3 


10 


37 




69 


1 


09 


618 


1 


25 


I 


52 


89 




80 


1. 


04 


83 


1 


32 


1 


87 


57 


1 


23 


1 


53 


41 



.99 874 



.96 513 



.75 61.0 



.87 1.09 77 .59 -71 6J 1.29 1.29 51 .71 1.06 34 



I 
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Table 4 



Correlations of Student Scales and Selected Other Characte:ris tics 
with Self-Report Indexes of Student Behavior 



Scale or Characteristic 


School 
Effort 


Self- 
Total 


■Reported Delinquency 

Drug Serious 


Parental education 


15* 


06 


03 


C'6 


Parental emphasis on education 


19* 


-22* 


-22* 


-17* 


Maternal role model (negative) 


-11* 


22* 


17* 


19* 


MAt"PT"nAT toIp mnHf»l ^ noQi 1 1 vp^ 

I ICL Im C L 1 1 CL JU k. \y JU C 11 IV^ U C JL \ r ^ ^ L. A. V C y 


11* 


-07* 


-05 


-08* 


Paternal role model (negative) 


-10* 


16* 


15* 


13* 


Paternal role model (positive) 


12* 


-07* 


-09* 


-05 


Attachment to parents 


22* 


-26* 


-31* 


-21* 


Negative peer influence 


-31* 


50* 


41* 


50* 


Parental supervision 


14* 


-27* 


-22* 


-24* 


Alienation 


-22* 


20* 


16* 


20* 


Attachment to school 


35* 


-34* 


-30* 


-34* 


Nonbelief in rules 


-22* 


27* 


20* 


30* 


Interpersonal competency 


22* 


-03 


m 


-OR* 


Involvement 


15* 


02 


-06* 


03 


Positive self-concept 


48* 


-24* 


-19* 


-29* 


X X* Ct ^ -1. CI JLm IVllWW JU C VA )^ C 


07* 


02 


04 


-01 


Rebellious autonomy 


-07* 


18* 


19* 


13* 


School punishments 


-22* 


30* 


17* 


30* 


School rewards 


15* 


-09* 


-14* 


-04 


Victimization 


-15* 


24* 


09* 


25* 


Self- reported reading ability 


20* 


-02 


-02 


-05* 


Self-reported school grades 


39* 


-11* 


-12* 


-14* 


Age 


-06* 


08* 


22* 


03 


Sex (male) 


-17* 


20* 


01 


25* 



Note : Decimals omitted. Sample sizes are very large, so even small correlations 

are significant. Size rather than significance of these coefficients should 
be attended to. • . 



p < .001 
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Table 5 



Correlations Among Student Measures of 
Family Background 



Scale 



1. Parental education (76) 

2. Parental emphasis on educ. 

3. Maternal role model (neg.) 

4. Maternal role model (pos.) 

5. Paternal role model (neg.) 

6. Paternal role model (pos.) 



Scale 



Note 



: Reliabilities shown in diagonal. Decimals omitted, 



9 


3 


4 


5 


6 


24 


07 


11 


-14 


17 


(46) 


-17 


30 


-20 


31 




(36) 


-11 


11 


-09 






(46) 


-07 


19 








(55) 


-40 



(71) 



Table 6 



Correlations Among Student Measures 
of Social Relations 



Scale 



Scale 



1. Attachment to parents 

2. Negative peer influence 

3. Parental supervision 



(61) 



-26 
(64) 



Note: Reliabilities shown in diagonal. Decimals omitted. 



21 
-25 
(27) 
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Table 7 



Correlations Among Measures of Student Attitudes 
and Social Development 



Scale 



Scale 


1 




3 


4 


5 


6 


7 


8 


1. Alienation 


(44) 


-53 


27 


-20 


-07 


-39 


-07 


19 


2. Attachment to school 




(75) 


-34 


27 


13 


41 


09 


-21 


3. Non-belief in rules 






(50) 


-13 


-02 


-31 


-01 


24 


4. Interpersonal competency 








(42) 


10 


39 


17 


09 


5. Involvement 










(64) 


11 


12 


-06 


6. Positive self-concept 












(63) 


12 


-07 


7. Practical knowledge 














(71) 


03 


8. Rebellious autonomy 
















(46) 



Note: Reliability coefficients shown in diagonal cells. Decimals omitted. 



Table 8 

Correlation of Student Scales with 
School Experiences 



Scale 

Scale 1 2 3 

1-. School punishments (54) 07 35 

2. School rewards (54) 15 

3 . Victimization (68) 



Note: Reliability coefficients shown in diagonal cells. 
Decimals are omitted. 



ERIC 
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Table 9 



Reliability Coefficients for the Individual-Level 
Teacher Scales and Raw Score Item Statistics 



Scale 



Number .^onstruction_^^ ^^^^^^^^^^^'^^^ 
of items ^ Alpha Mean^SD__Alpha 



Prointegration attitude 

Job satisfaction 

Interaction with students 

Type A sanctions 

Type B sanctions 

Victimization 

Classroom disruption 

Low expectations 

Openness to student 
suggestions 

Professional development \ 
Nonauthoritarian attitude 



4 

3 

6 

5 

5 

8 

2 

2 



8 
3 



11.53 
8.45 

14.20 
-.05 

13.81 
1.24 
4.52 



-.21 
7.57 



2.81 
1.64 
4.43 
2.90 
3.08 
1.45 
1.22 



62.43 42.89 
.05 1.64 



4.82 
2.21 



.67 

.78 

.69 

.52 

.58 

.67 

.70 

.53 

.45 

.76 
.56 



11.56 
8.42 

13.79 
.09 

13.42 
1.23 
4.60 



2.88 

1.70 

4.20 

2.82 

3.08 

1.45. 

1.38 



65.06 44.47 

-.10 1.54 

-.39 4.67 

7.43 2.17 



^N's range from 555 to 642 due to nonresponse to some items. 
^N's range from 555 to 643 due to nonresponse to some items 



.69 

.80 

.67 

.47 

.60 

.67 

.78 

.57 

.38 

.74 
.54 



r 
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• Table 10 








■ ■ ■ ; 




Correlations Among Individual-Level Teacher Scales 














(N=1112- 


1265) 










Scale Prointeg. 


Job Sat. Interact 


. Type A Type B Victim. Disrup. 


Low Exp. 


Openness 


Prof .Dvt. 


Nonauth 


Prointegration 
attitude 


06 12* 


-09 


04 -09 -10* 


-07 


15* 


08 


29* 


Job satisfaction 


28* 


-13* 


00 -20* -34* 


-24* 


11* 


28* 


13* 


Interaction with 
















students 


— 


-02 


11* 01 -15* 


-12* 


24* 


28* 


11* 


Type A sanctions 






16* 19* 29* 


12* 


-03 


-03 


-26* 


Type B sanctions 






08 12* 


12* 


09 


17* 


02 


Victimization 




* 


38* 


25* 


-02 


-07 


-18* 


Classroom 
















disruption 








43* 


-02 


-12* 


-16* 


Low expectations 








— 


01 


05 


-07 


Openness to 
















stud. sug. 










— 


19* 


14* 


Professional 
















develop. 












— 


10* 


Nonauthoritarian 














— 


Note: N's vary from 1112 to 


1265 due to teacher 


nonresponse 


to some items. 










* p<.001 
















>— • 

00 
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• 


• 





Table H 
Correlations of Individual-Level 
Teacher Scales with Selected 
Teacher Characteristics 



Scale 


Sax 
(female) 


Non- 
white 


Use of 

home-based 

reinforcement 


Item 
non-response 
index 


Range 
of N 


Prointegration attitudes 


10^ 


21* 


04 


04 


1204-1241 


Job satisfaction 


01 


07 


06 


-08 


1274-1315 


Interaction with students 


-05 


13* 


17* 


06 


1246-1289 


Type A sanctions 


-10* 


-02 


07, 


-01 


1169-1-206 


Type B sanctions 


15* 


02 


35* 




1192-1230 


Victimization 


-15* 


-18* 


08 


-11* 


1175-1212 


Classroom disruption 


-04 


-15* 


12* 


-02 


1183-1223 


Low expectations 


oil 


-02 


11* 


02 


1168-1205 


Openness to student 
influence 


06 


09 


09 


02 


1128-1164 


Professional development 


13* 


27* 


16* 


10* 


"1190-1223 


Nonauthoritarian attitude 


15* 


01 


-04 


08 


1207-1245 



Note: Decimals omitted 
*p <.001 

^High scorers on the scale measuring use of type B sanctions failed to ^"pond 
to many items intended to measure type A sanctions (r=.44), f "^^"f .^f ^ '^"^ 
were using the questionnaire as a checklist rather than carefully marking a 
response for each item. 
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Table 12 



Reliabilities of the School Climate Measures Based on Aggregated 

Student Reports 





Alpha 

— ■ ■ ■ , — — ^1 


Number of 
items 


Community Crime 


. 57 


3 


Gangs in School 


. 80 


2 


Safety 


.92 


13 


Disruption 


.42 


4 


Individualized Instruction 


.58 


2 


Student Disrespect 


.78 


3 


Student^Teacher Interaction 


.60 


2 


Planning and Action 


.65 


3 


Fairness 


.62 


3 


Clarity 


.64 


4 


Student Influence 


.62 


6 


Grouping 


.55 


3 
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Table 13 

Correlations Among School Climate Measures Based on Aggregated Student Reports 



Scale 



Scale 



1. Community Crime .39* -.24 .14 .24 -.03 -.22 

2. Gangs in School --01 -.02 -.12 .01 -.33 

3. Safety -^3* -.40* -.13 -.02 

4. Disruption -36* .66* -.36* 



Individualized Instruction -.22 .01 

-.59* 



5 

6. Student Disrespect 

7. Student-Teacher Interaction 

8. Planning and Action 

9. Fairness 

10. Clarity 

11. Student Influence 

12. Grouping 



8 


9 


10 


11 


12 


. uo 




- . 16 


- .04 


. 13 


.30 


-.18 


-.26 


-.30 


.18 


.03 


-.06 


.10 


-.27 


-.22 


.33 


-.49* 


-.16 


-.20 


.55* 


.02 


.02 


.02 


.37* 


.17 


.39* 


-.60* 


-.30 


-.53* 


.33 


.45* 


.42* 


.35* 


.43* 


-.28 




.50* 


.68* 


.62* 


-.39* 






.45* 


.56* 
.48* 


-.37* 

-.24 

-.22 



Note.— N's: range from 52 to 65 schools, depending on the availability of items to score these scales. 
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Table 14 



Reliabilities:.of the School Climate Measures 
Based pn Aggregate Teacher Reports 

Number of 

Scale Alpha Items 

Involvement of Parents and" Community^ 

Individualized-^Instruction and Grading 

Resources for Instruction 

Integration vs. Segregation by Ability 
or Conduct 

School Race Relations 
Interaction with Students 
Teaching Staff Commitment 
Use of Grades as a Sanction 
Success Opportunities 
Staff Morale (vs. Alienation) 
Planning and Action 
Perceptions of Disriii^tion 
Student Influence 
Smooth Administration 
Professional Development 



Note: N=48 to 50 schools (based on responses of over 1,100 teachers). 
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.80 
.60 
.86 



6 
4 
4 



.55 
.77 
.80 
.82 
.84 
.60 
.90 
.87 
.93 
.81 
.92 
.86 



6 
2 
6 
2 
2 
3 
11 
10 
13 

5 

12 
8 



Measures 

Table ^ 

Item Content of Individual-Level Student Scales<a> 



Parental Educa tion 

20. HOW far did your father (or stepfather) go in school? 

21. HOW far did your mother (or stepmother) go in school? 

£ai:ental_Em£hasis_on_Educat 

23. DO your parents -ant you to go to college some day? 
28. My parents keep close track of how well I am doing in 
school. 

26. (Father) helps me with my homework. 

27. (Mother) helps me with my homework. 

^ternal_Role_Model_iNeaativei 
26. (Mother) drinks too much. 

26. (Mother) gets in trouble with the police. 
26. (Mother) spends most of her money on herseir. 
26. (Mother) gets mad a lot. 
- 26. (Motlier) spends time with her friends away from the 

house. 



Maternal Rol e_Model_iPositive| 

26. (Mother) helps me with personal problems. 

26. (Mother) is a hard woc^ker. 

26. (Mother) gives me money when I need it. 

26. (Mother) goes to work every day. 

Paternai_Role_Model_iNeaativel 
26. (Father) drinks too much. 

26. (F-^ther) gets in trouble with the police. 
26. (Father) spends most of his money on herself. 
26. (Father) gets mad a lot. 

26. (Father) spends time with his friends away from the 



house. 



<a>Numbers indicate position of the item in the quastion- 
naire. 
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Measures 



Table A (cont«) 



Paternal .Rol e ^odel^ ^Xg2Siti^gl 

26. (Father) helps me with personal problems* 

26. (Father) is a hard worker* 

Zbm (Father) gives me money when I need it* 

26* (Father) goes to work every day* 



Attachme nt to Parents 

29* How much do you want to be like the kind of person 

your mother (or stepmother) is? 
30* How close do you feel to your parents (or guardians)? 
31* How much do you want to be like the kind of person 

your father (or stepfather) is? 
32* All in all? how much do you like your parents? 
68* I would not care if my parents were a little disap* 

pointed in me* 
69* I have lots of respect for my parents* 



Negat i ve .Peer _ Inf 1 uence 

43* Most of my friends think getting good grades is 

important* (-) 
43* Most of my friends think school is a pain* 
43* My friends often try to get me to do things the 

teacher doesn*t like* 
44* (Best friend) is interested in school* (•) 
44* (Best friend) attends classes regularly* (•} 
44* (Best friend) plans to go to college* (•) 
44* (Best friend) belongs to a gang* 
44* (Best friend) gets in trouble with the police* 
45* How many of your friends have been picked up by the 

pol ice? 



Parental Supgrvi s ion 

23* My parents almost always 

am doing* 
69* As far as my parents are 

free to come and go as I 

Al i ena t i on 

64* Teachers here care about the students* (-) 
64* I feel like I belong in this school* (-) 
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know where I am and what I 

concernedf I am pretty much 
please* 
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Measures 



Table ^ (cont.) 

69. I feel no one really cares much about what happens to 

70. These oa/s I get the feeling that I'm just not a part 
of things. 



Attachment to_School. 



62. 
63. 
63. 
63. 
63. 
63. 
64. 
64. 
62. 
70. 



(HOM 

(How 
4 How 
(How 
(How 
(How 



important is) what 
do you feel about) 
feel 
feel 
feel 
feel 



think about you? 



are taking? 



the teachers 
this school? 
the principal? 
the classes you 
the teacners? 
the counselors? 
of respect for my teachers, 
makes me like to learn. 
(How important is) the grade you get at school? 
In classes I am learning the things I need to know. 



do 
do 
do 
do 



you 
you 
you 
you 

1 have lots 
This school 



about ) 
about ) 
about ) 
about ) 



Bel i ef _ i.n_Rules 

69. It is an right to get around the law if you can. (-) 

69. People who leave things around deserve it if they get 

taken. (-) , , 

70. Taking things from stores doesn't hurt anyone. (--) 
70. It is O.K.. to. take advantage of a chump or a sucker. 

70. Teichers who get hassled by students usually had it 

coming. (-) i „ 

68. I do not have much to lose by causing trouale in 

school . 



iQ termers on al_Com£etencx 

69. I have a clear picture of what I am like as a person. 
69. I know how to get along with teachers. 

69. If I want to» I can explain things well. 

70. 1 find it easy to talk to all kinds of oeople. 
70. My friends regard me as a person with good sense. 



I nvol vement 

34. Which of the following things have you spent time on 
this school term? (12-itefl list follows) 

35. How much timet on the averagef do you spend doing 
homework? . 

36. Did you do any work for pay last week, not counting 
work around the house? 37. Do you have a regular 
part-time or full-time job? 
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Measures 



Table A (cont«) 



Posi tive Self^Concegt 



?• How satisfied are you with the way you are doing in 
school ? 

69. Sometimes I think I am no good at all* 

70* I feel I do not have much to be proud of* (-) 

70* I like mysel f • 

9* (Other students see me as) a loser* (•) 

68* I am the kind of oersbn who will always be able to 

make it if I try* 
9* (Other students see me as) a good student* 
9* (Other students see me as) a trouble aiaker* (-) 
9* (Other students see me as) successful* 
68* My teachers think that I an a slow learner* (-) 
68* I 30 not mind stealing from someone—that is just the 

k i nd of person I am* ( - ) 
68* I am not the kind of person you would expect to get 

in trouble with the law* 



Pr act te al ><no»<1 ed g e 

73* Do you know how to: (seven competency items) 



Rebel 1 ious Autonomy 

69* I don't like anybody telling me what to do* 
70* Whether or not I spend time on homework is my own 
Dusi ness* 

70* I should not have to explain to anyone how I spend my 
money* 



ERIC 



School_Effort 

8* Compared to other studentst how hard do you work in 
school? 

33* 1 turn my homework in on time* 
38* My schoolwork is messy* (-) 

38* I don't bother with homework or class assignments* 
(-) 

38* If a teacher gives a lot of homework? I try to finish 
all of it* 
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Measures 



Table A (cont^) 



School Nona t tend a nee 

13, In the last four -eeks. ho- «.any days old you cut 
1.. H^rofUh r'you cut one or .ore of your classes. 

Seif^fiortel_0elinauencj[_il2i.aLi 

^ -c mark -ves" or "no" to 19 kinds of behav- 
^6-49* Respondents mark yes or 

ior* 

46-^^9. In the last year have you ... 
...smoked cigarettes? 

...drunk beer, wine, or "hard" liquor? 

...smokea marijuana (grass, P°t» Qanj^); 

...gone to school when you were drunk or hiQh on so-ne 

drugs? 

...taken some other drugs?. 
SeifzBjeBO£ted_::Serious::_0elinau£nc3t 

^^-^Il.J^r^p^LlrSarag^^^ordntro;;; property .e.on.ing 

...pur'pore^riUed or destroyed f^ej P-P^^^ jot 
belonging to you, not counting family or school 

...l[T/nVr tried to steal something worth more than 

...carried a hidden weapon other than a plain pocket 
knife? 

...been involved in gang fights? adult 
...hit or threatened to hit a teacher or other aduit 

...fakenTcIr for a ride (or drive) without the own- 

...rs;d J^e^rTtrcng-arm methods to get money or 

...T.Tel l7lrter.^sl.^ things worth less than 

...I'tolen or tr i ed-to steal something 

as someone's coat from a classroom, locker, or 
cafeteria, or a booK from the ..^ to 

...broken or tried to, break into a building or car to 
steal something or just to look around? 
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Measures 

Table A (cont«) 



School gyflispggnts 

59. Did you have to stay after school as a punishment? 
59. Did you get an extra assiqnfnent as a punishment? 
59« Was your grade lowered on an assignment as a punish- 
ment? 

59m Were you sent out of class for punishment? 



Reward s .Ct^g^gA Cont ain s two i terns that shoul d be remgyeci*) 

5lm Students get to help othar students. 

58. Teachers say nice things about my classwork. 

59« Did you jet to do something special as a reward? 

59. Did you win an award or prize because of your work in 
school? 

59« Did you help win an award or a prize for your group 
or class because of your work in school? 

6l« Students who are well-behaved in this school get spe- 
cial treatment. 

V ict imi za ti on 5 
60« Seven-^item list* See questionnaire. 



Irjyal^idi ty 

I have never disliked anyone. (T) 

It is easy to get along with nasty people. (T) 

1 so met i mes get angry* (F) 

I like to have fun. (F) 

I read several whple books every day. (T) 
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Tdbl e 9 

Individual-Level Teacher Scales 

Pro- Int egration, Attitude 

Most black students ara better off in all-black schools. 

Most'lhite students are better off in all-white schools. 

The^aiount of prejudice against minority groups in this 
country is greatly exaggerated. (-) K=.i^nrp 
Students should not be bussed to achieve racial balance. 

(-) 

job_Satxif action 

How do you like your job? „k-> 
HOW much of the time do you feel satisfied with your job? 
HOW much do you think you like your job compared with 
other people? 

i!2tSil§£ii2!!_!iitl3_Students^ 

In tne past two weeks have any students come to you to 

-TSk your advice on some prolbem they were haveing out- 
si deofcl ass? ... 

HOW Often do you engage in the following activities with 

students: 

. . . tutoring individual students before or after 

school . . 1 ,^«. i 

. . . working with students on extracurricular activ 

i t i es . 

. . . taking students on field trips. 
. . . going to gamest dancest and other student 
activities. . 
. . . discussing students* personal problems with 

them. 

In your dealings with misbehaving students how often do 
• you do the following things? 

... send them out of class. 

... use or threaten to use physical punishment. 
. . . lower their grades if misconduct is repeated. 
... reprimand the student in tne class. 
When a student misbehaves in my class, I sometimes lower 
his or her grade. 
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Table B (cont.) 

Tyge 8 Sanctions 

In your dealings with misbehaving students how often do 
you do the following things? 

• • • give additional school work* 

• • • give privileges to increase positive behavior* 

• • • w i thdraw priv i 1 eges for mi sccnduct* 

• • • cal 1 a parent • 

• • • get help from a counselor* 

• • • refer the student to a special prograii* 

Vict i mizat i o n 

How many times in the past month have the following hap- 
oenecj to you personally in this school? 

• • • damage to personal property worth less than 

• • • damage to. personal property worth more than 

• • • theft of personal property worth less than llOm 

• • • theft of personal property worth more than tlO. 

• • • was physically attacked but not seriously 
enough to see a doctor* 

• • • received obscene remarks or gestures from a 
student* 

• • • was threatened i n remark s by ^ student. 

• • • had a weapon pulled jn me* 

C 1 assroo m 3|sruptron 

How much of your time in the classroom is directed to 
coping with disruptive student behavior? 

How much does the behavior of some students in your 
classroom (talking* fightingt etc#) keep you from 
teach i ng? 

Low Expectations 

Of the students you teacht what percentage would you say 

are 1 ow abi 1 i ty? 
Of the students you teacht what percentage are behavior 

proDlems? 
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Table B (cont.) 



Openness to_Studgnt_Suaaastions 

I often change my lesson plans based on student sugqes- 

Studen^r should have a lot to say about how the school is 
run* 

P£2lSiii2Qll-2Svelogment 

HOW often do you attend professional develooment courses 

that are half a day or more in length? 
HOW much training have you had in teaching mt3thods and 

curriculum content in the last 12 months? 
How mucn training have you had in interpersonal or inter- 

iroup 'relations in the past 12 months? ^ 
in some school years, a teacher learns a lot aoout educa- 
tion, while in other years a teacher doesn't learn 
much. This year, have you learned much about. 
... new materials, new kinds of texts, supplemen- 
tary materials? 
. . . theories of teaching reading? 
. . . effective methods of maintaining discipline. 
. . . how to handle disruptive students: 
... how better to deal with heterogenous classes? 

If a pupil uses obscene or profane language in school it 
should be considered a moral offense. (-) 

A few pupils are just young hoodlums and should be 

treated accordingly. (-) ct:^^*.,.,^ 

The threat or use of physical punish,nent is an effective 
way of dealing with misbehaving students. (-) 
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Table C 



Item Content of Student School Climate Scales<a> 



C ommunity Crime an d Gangs 



33. Are there any gangs in the neighborhood where you 
I i ve? 

33* Do gang members try to get you to join their ganjs? 
33. In the last year has either of your parents been rob- 
bed on the streets of your nei ghborhood? 

Gangs in School 

33. Are there gang members in your school? 
33« Do gang members cause a lot of trouble in your 



65. Do you usually stay away from any of the following 
places because someone might hurt or bother you 
there? 

--The shortest way to school (-) 

— Any entrances into tha school (-) 

— Any hallways or stairs in the school (-) 

— Parts of the school cafeteria (-) 

— Any school restrooms ( -) 

— Other places inside school building (-) 

--Other places on the school grounds (-) 

66. In this term in school* have you: 

— Had to fight to protect yourself? (-) 

— Seen a teacher threatened by a student? {-) 

— Seen a teacher hit or attacked by a student? (-) 

67. How often do you feel safe while in your school 
building? 

67« How often are you afraid ttiat someone will hurt or 

oother you at school? {-) 
67. HOW often are you afraid that someone will hurt or 

bother you on the way to or from school? (-) 



<a>Numbers indicate position of the item in the question 
nai re« 



school ? 



Safety 
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Table C (coit.) 



Pi srupti on 

(Note that this scale is partly redundant with the previous 



one • ) 1 1: t 

66. (Have you) had to fight to protect 1°^''^^^ 

66. Have you) *een a teacher threatened by a student? 

66. (Have you) seen a teacher hit or attacked by a stu 

66. %all you) been in a class that was totally stopped 
by a disruptive student? 

Indi yi d ualLie<l-iDllIl^^iSn 

53. I have a learning plan made just for me. 
53. I can work at my own speed in class. 

Student_DisresBect_lDisres2ect_for_St 

51. Students are treated like children here. 
61. Teachers treat students with respect. (-) 
61. Teachers do things that make students feel pu. 
down* 

Student-Ieacher_ Interaction 

58. I talk to some of my teachers about thinjs other than 

homework. ^ ■ ^ c ^i=.cc 

58. Teachers help me with school work outside of class. 

PI an nina_and_Action 

56. It is hard to change the way things are dona in this 

52. The°tlachers and principal in this school T»ake plans 
to solve proolems. 
^ 52. This school hardly ever tries anythinq new. 



Fa i rness 



51. The schciol rules are fair. 

51. The punishment for breaking the rules is tne same no 

matter who you are. 
57. The principal is fair. 
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Measures 

Table C (cont.) 

CI ar i ty 

51. Everyone knows what the school rules are. 
56« The teachers let the students know what they expect 
of them* 

51. The principal runs the school with a firm hand. 

55. The principal lets the stulents know what he or she 
expects of them* 

Student Influence 

52. Students have little S3y in how this school is run. 
(-) 

51. Students can get an unfair rule changes. 

5Z. The student government makas important decisions. 

52. Teachers sometimes change their lesson plans because 
of student suggestions. 

52. Students are seldom asked to help solve a problem the 
school i s hav i ng. ( - ) 

56. Students have helped to mvike the school rul^s. 

Grouping 

56. Students of different races usually end up in differ- 
ent classes. 

56. This school has special classes for slow learners. 
56. There are special classes for trouble makerso 
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Ttem-Content of School-Level Scales 
Based on Aggregate Teacher Reports 



Resources for Instruction 

a. This school supplies me with the material and equipment I need for 
teaching. 

b. This school building has the space and physical arrangements needed 
to conduct the kinds of programs we need. 

c. The school's learning program extends to settings beyond the school 
building for most students. 

d. Teachers and students are able to get the instructional materials 
they need at the time they are needed. 



Involvement of Parents and Community 

a. How much influence on school policies or practices does a PTO have? 

b. How often do... parents help to decide about new school programs? 

c. How often do... parents serve as tutors or aides in the classroom? 

d. How often (is) ... community involvement ... sought in developing the 
school's goals? 

e. (How well do) parents and teachers (get along at your school)? 

f. Parents and' the community are receptive to new ideas. 



Teaching Staff Commitment 

a. (Are the teaching faculty) involved? 

b. (Are the teaching faculty) uncommitted? (-) 

Staff Morale (vs. Alienation) 

a. Students here don't really care about the school. (-) 

b. Our problems in this school are so big that it is unrealistic to 
expect teachers to make much of a dent in them. ( ) 

c I feel my ideas are listened to and used in this school, 

d! I want to continue working with the kind of students I have now. 

e. (Is the teaching faculty) apathetic? (-) 

f. (Is the teaching faculty) cohesive? ' 

g. (Is the teaching faculty) enthusiastic? 

h. (Is the teaching faculty) frustrated? (-) 

i. (Is the teaching faculty) satisfied? 
j. (Is the teaching faculty) tense? (-) 
k. (Is the teaching faculty) unappreciated? (-) 



e 



Planning and Action 

a. How often do you work on a planning committee with other teachers 
or administrators from your school? 

b. xThe principal encourages experimentation in teaching. 

c. Teacher evaluation is used in improving teacher performance. 

d. (The principal is) planful. 

e. (The principal is) progressive. 

f. (The teaching faculty are) ^ conservative . (-) 

g. (The teaching faculty are) innovative. 

h. (The teaching faculty are) open to change. 

i. (The teaching faculty are) traditional. (-) -135- 
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Table D ' cont* 

Smooth Administration 



a. Simple non- time-consuming procedures exist for the acquisition and 
I use of resources. 
hi (How well do) teachers and administrators (get along at your school)? 

c. Administratbrs and teachers collaborate toward making the school 
run effectively. 

d. There is little teacher-administrator tension in this school. 

e. Our principal is a good representative of our school before the 
superintendent . 

f. The principal is aware of and lets staff members and students know 
when they have done something particularly well. 

g. Teachers or students can arrange to deviate from the prescribed 
program of the school. 

h. Teachers feel free to communicate with the principal. 

i. The administration is supportive of teachers. 

j. It is hard to change established procedures here. (-) 
k. (The principal is) informal. 
1. (The principal is) open. 

Individualized Instruction and Grading 

a. Mj^ students mostly work according to individualized learning plans. 

b. Students in my classes generally receive grades based on improvement 
in their performance rather than in comparison with other students. 

c. Grades in my classes are typically based on the curve. (-) 

d. Grades in this school are typically based on the curve. (*) 



ERIC 



School Race Relations 

a. (How well do) students of different races (Ret along at your school)? 

b. (How well do) students of different nationalities (get along 
at your school) ? 

Interaction with Students 

a. In the past two weeks have any students come to you to ask your 
advice on some problem they were having outside of class? 

b. (How often do you engage in) tutoring individual students before 
or after school? 

c. (How often do you engage in) working with students on extracurricular 
activities? 

d. (How often do you engage in) going to games, dances and othev 
student activities? 

e. (How often do you engage in) discussing students* personal problems 
with them? 

f. (How well do) teachers and students (get along at your school)? 



Integration vs. Segregation by Ability or Conduct 

Students of 
Most of my .5 
ability. (.) 



a. 
b. 



c. 
d. 
e. 
f . 



S^Eudents of mixed ability work together in small groups in my class. 
Most of my students are assigned to my classes on the basis of their 
ability. (-) 

All students in my classroom are of the same general ability level. (-) 
This school has specieil classes for slow learners. (-) 
This school has special classes for high ability students. (-) 
In this school there are special classes for students who repeatedly 
misbehave. ('-) f j 



misbehave. (-) 
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Table D cont. 

Student Influence 



a. 
b. 

c. 
d. 
e. 



I often change my lesson plans based on student suggestions 
Teachers and their students work together to make rules governing 
behavior in the classroom. 

Students can get an unfair school rule changed. 

Students help to make,, the school rules. ■ „i run 

Students should have a lot to say about how the school is run. 



Professional Development 



a. 



e. 
f . 



h. 



How often do you attend profes 
half day or more in length? 
(How much in-service training 
or curriculum content (in the 
(How much in-service. . .had in) 
(in the last 12 months)? 
(Have you learned much about) 
supplementary materials? 
(Have you learned much about) 
(Have you learned much about) 
discipline? 

(Have you learned much about) 
(Have you learned much about) 
classes? 



sional development courses that are a 

have you had in) teaching methods 
last 12 months)? 
interpersonal or intergroup relations 

new materials, new kinds of texts, 

theories of teaching reading? 
effective methods of maintaining 

how to handle disruptive students? 
how better to deal with heterogeneous 



Perceptions of Disruption or Lack of Safety 



a. 
b. 
c. 

d. 



(In your opinion, how much of a problem are vandalism, personal 
attacks and theft) in your school? 

How much of your time in the classroom is directed to coping with 
disruptive student behavior? 

How much does the behavior of some students in your classroom 

(talking, fighting, etc.) keep you fron teaching? ^ 

Since school started this year, how many times did you hesitate to 

confront misbehaving students for fear of your own safety. 

(How safe is) your classroom while teaching? 

(How safe are) empty classrooms? 

(How safe are) hallways and stairs? 

(How safe is) the cafeteria? 

(How safe are) the restrooms used by students? 
(How safe is the) locker room or gym? 
(,How safe is the) parking lot? 

(How safe is it) elsewhere outside on school grounds? 
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Success Opportunities 

a- In this school , students who do w.ell often get special privileges. 

b. Any student can earn an A in my class . 

c. Some students in my classes earn mostly D's and F*s because they 
cannot keep up with other students. (-) 

Use of Grades as a Sanction 

a. When a student misbehaves in my class , I sometimes lower his or her 
grade . 

b. (In... how often do you) lower their grades if misconduct is repeated? 
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Overvi of 



Interim Results for the 
Program 



Alternative Education 



Denise C Gottfredson 



The School Action Effec- 
tiveness Study (SA6S) has 
completed its first year of 
evaluation activities. A 
summary or ovHrview of the 
status of the evaluation and 
some inter i m--or format i ve-- 
-evaluation results are pro- 
vided in this chapter as a 
quick guide to the 17 action 
projects* Each of these 
projects is di scussed in 
more detail in Part 2 of 
this report? and readers 
should consult those 
descriptions for more detail 
about a specific project* 

Conclusive statements are 
seldom possible at this 
stacje in 3ny large-scale 
eval uation. Oevel oping* 
implementing and evaluating 
social programs takes time. 
In jeneralf it is too early 
to reach summative judgments 
about the effectiveness of 
these orojectsf and few will 
be found in these pages. 
Information presented here 
is in the nature of forma- 
tive evaluation; it should 



I am grateful for the com- 
ments on a draft of this 
chapter by Phyllis BetZt Viic 
Cooper? Deborah Dan i el s t 
Gary 0. Gottfredsont Joe 
Nathant Jane St. Johnt Dave 
ReisSf and Sally Wisotzkey. 
Opinions expressed aret how- 
evert solely my own. This 
report coverst for the most 
partf the period ending 
Augustf 1931. 



be 



read in that spirit. 



Comments about the nature 
of the interventions being 
attemptedf the progress made 
towards th^ir implementa- 
tion, and the vigor and 
rigor with which their eval- 
uation is being pursued aret 
hawevert timely. In this 
chaptert as in most of the 
project descriptions found 
in Part 2f frank 

attempts are made to give 
feedback about the projects. 
This feedback is intended to 
provi de rei nf or cement for 
what appear to be project 
strengthSf as well as impe- 
tus to overcome some weak- 
nesses as we see them. 

i IS e1 § SSIli i 2 Q 

The IT alternative educa- 
tion projects are in varying 
states of i mpl ementati on. 
Some are still floundering 
with start-up efforts and 
with unclear .]oals and oth- 
ers are extending or repli- 
cating efforts they have 
tried elsewhere or at an 
earlier time. ^ brief char- 
acterization of each project 
is presented in ^^able I. 
See the i nd i v idual project 
descriptions for nore com- 
prehensive accounts. 

Some common themes arise 
from a reading of the more 
detailed project descrip- 
tions in Part 2 of this ; 
report. Firstt a difficulty 
experienced by many of the 
projects relates to the 
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Overview 



timing of the qrant dward 
notification* For most 
projectSf notification came 
just at the beginning of the 
school yeart tV^^ving them in 
a state of uncertainty* 
Many of "thiT projects thus 
entered the 1980-31 school 
year without planst and 
unable to begin work immedi* 
atelyt but feeling pressure 
to do so* The long lead 
times required for adminis- 
trative decision making 
(e«g«f in the Chicago Board 
of Education project)* or 
the inflexibility of school, 
system arrangements t often 
thwarted i mp 1 ementat i on 
further* 

In a few cases? grant 
awards were made in wintert 
between semesters* Againt 
this appears to be an awk- 
ward time to begin a 
school-based project* Pre- 
sumably notification of 
grant awards would be best 
made near the close of the 
academic year preceding 
project start-upf if not 
earliert to facilitate proj- 
ect planning and smooth 
impi ementdt i on* 

Secondf school system 
changes* such as grade 
structure reorganization* a 
change of f^dmi n i s t rators* or 
reductions in staff size* 
create serious problems for 
project impi ementers* Some- 
times* as in the Kalamazoo 
project? interventions were 
planned in collaboration 
with personnel who were no 
longer there when the proj- 
ect began operation* 

Third* ambiguity about 
project staffing or diffi- 
culties with staff turnover 
can impede implementation* 



For example* the designation 
of project directors wis 
delayed in both the Chicago 
Board of education and St. 
Paul projects. In the case 
of the Chicago Board proj- 
ect* ponderous administra- 
tive machinery seems to be 
the cause of this delay* 
The Sronxf Hayward* Houston* 
and Harlem project directors 
were replaced seye^ral months 
into the project* and other 
projects have experienced 
i nstibi 1 i ty in their staff- 
ing patterns. These prob- 
lems naturally cause diffi- 
culties in implementation* 

Fourth* projects differ 
in the clarity of their 
Implementation plans* Proj- 
ects without clear plans* 
and projects that do not 
engage in systematic plan- 
ning* must struggle harder 
for success* 

Finally* evaluation taxes 
the resources and patience 
of most projects* A«nbigui- 
ties in the RFP regarding 
the nature of the evaluation 
and the level of resources 
projects would have to allo- 
cate to evaluation activi- 
ties left projects unpre- 
pared for* and sometimes 
bitter about* the intensive 
activities they were 
expected to undertake* Spe- 
cifically? few projects 
expected evaluation consid- 
erations to influence the 
selection of students for 
their interventions* Estab- 
lishing and maintaining a 
rigorous evaluation design 
without a prior agreement 
among all affected actors 
requires intensive negotia- 
tion and careful monitoring* 
In addition? some projects 
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were unprepared to provide 
individual-level data even 
for youths receiving project 
serviceSf and others had 
difficulty providing data 
for comparison students. 
Furthermore^ the SAES stu- 
dent survey was also diffi- 
cult to implement? not only 
because of the sensitive 
nature of many questions? 
but also because its admin- 
istration required a large 
commitment of personnel 
resources? because some 
projects expected to conduct 
a different kind of evalua- 
tion? and because of inade- 
quate or unrealistic budget- 
ary planning for evaluation. 
The response rates shown in 
Table 2 are in part an indi- 
cator of the level of 
resources projects were 
willing to allocate to this 
important evaluation activ- 
ity • 

These observations have 
five implications for tne 
future efforts of these 17 
projects? and for similar 
projects attempted in the 
future: 

I. .Notification of fund- 
ing should be made before 
the end of the academic year 
preceding the anticipated 
act i vi ty • 

Projects in school 
systems undergoing other 
major administrative or 
structural changes should be 
avoi ded« 

3* Staffing plans should 
be well considered and made 
in advance of project 
star t-up* 



^. Project impl ementers 
should be proactive in the 
development of plans^ The 
importance of clear plans 
cjn scarcely be overstated. 

5. Evaluation require- 
ments and guidelines for the 
level of resources to be 
allocated to the evaluation 
should be specified in the 
RFP; evidence of tho proj- 
ect's intent to and ability 
to comply with the guide- 
lines should be required* 
Projects should be site-vi- 
sited prior to fundinq? and 
detailed written agreements 
regarding access to informa- 
tion and experimental 
designs should oe formulated 
at that time# The content 
of these agreements shoul d 
form one oasis for making 
decisions about funding. 

Overv i ew of _Eval_ua tion 
^^ssuTtilfor^thi^fntire^P 

□nly rarely were the con- 
ditions necessary for making 
inferences about project 
effectiveness ( see Chapter 
^) met during the first pro- 
gram year. The evaluation 
designs for the first year 
,ire summarized in Table 3. 
Some projects used the first 
year as a planning and 
staff-training period and 
hence did not start provid- 
ing services. Some provided 
services? but only on a 
trial basis* Others pro- 
vided services for the bet- 
ter part of the year but had 
already selected the^ stu- 
dents b/ the time the 
nat ional eval uat ion staff 
contacted them to set up 
sel ection procedures that 
would result in a defensible 
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experimental or 
qudsi-exper i mental desija^ 
The evaluation was funded 
and began work on 29 Septem- 
ber I9ai« A longer start-up 
period for the evaluation? 
with the opportunity to work 
with projects before they 
began implementati on? would 
have been rJes i rabl e« 

£l2iects_Im2lementin3_True 
Exper iment s 

Two projects attempted 
experiments during the 
Spring semester. Peer Cul- 
ture Development (PCD) and 
the Chicacjo Board of Educa- 
tion's Project RETAIN ran- 
domly assigned students to 
treatment and control 
groups. Although both proj- 
ects met with some obstacles 
as they implemented their 
desiqnsf the experience was 
valuable because they were 
able to plan strategies to 
overcome these obstacles for 
the following semester. 

£§SL«.£iiI t ur e_pe ve2o2men t • 
PCD developed a Ta7ge"*enougn 
pool of referrals who volun- 
teered for PCD classes to 
r andoml y ass i gn the students 
to treatment and control 
groups. But for a few class 
per i odSf all students in the 
pool had to be placed in the 
treatment group. And in 
some classeSf student 
as s i gnments were made non- 
randomly^ persons who had 
been designated as "treat- 
ment" students did not par- 
ticipate in the PCD classest 
and some control students 
received PCD services. The 
effective for the exper- 

iment was reduced considera- 
bly for these reasonst and 
data for certain classes had 
to be excluded from the 



study? weaken i ng the exper i - 
mental design. For statis- 
tical purposes persons were 
treated as if they had 
experienced the experimental 
condition to which they were 
assigned in the randomiza- 
tion process. The results 
of the experi ment 3ppl y onl y 
to those classrooms included 
in the experiment. In addi- 
tion to these limitationst 
PCD discovered that random 
ass ignment altered the com- 
position of th^ir classes 
because the grouo 1 eader s 
had 1 ess control over the 
designation of participants. 
The project personnel t on 
the basis of their first 
semester experiencet planned 
and implemented procedures 
for the following semester 
tnat woul d increase the pool 
of eligibles and ensure a 
more desirable mix of stu- 
dents in the group. 

Despite these 1 imita- 
tionSf useful information? 
described in the project 
summary by St. John (this 
volume)f was obtained from 
this experiment. The PCD 
treatment as impl e men tea 
reduced disciplinary infrac- 
tions? and had positive 
effects on interpersonal 
competency and practical 
knowledge^ the treatment had 
an unanticipated negative 
side-effect on oelief in 
conventional rules* This 
project's exemplary evalua- 
tion is continuing and is 
bei ng strengthened in the 
1931-82 school year. 

tion. C3E had little trou- 
ble obtaining amole refer- 
rals fcir its program? but 
the randomization process 
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broke down in some of its 
schools? resultinq in non- 
equivalent comparison groups 
in those schools^ ThuSf 
although post randomization 
checks did not detect fail- 
ures of randomization in 
most CBE schools? the way 
randomization was conducted 
on-site weakened the design* 

Additional problems 
involved the questionnaire 
admi n i strat i on? an i mportan t 
design consideration because 
even a true experiment is 
diminished in value without 
sound measurement of the 
outcomes of interest* Low 
response rates for the stu- 
dents in the study resulted 
in small N*s and biased sam- 
ples« To complicate matters 
further? CB£ censored most 
self-report del inquency 
items in the quest ionnai re« 
Finally? only two months 
elapsed between the date the 
students Degan receiving 
RETAIN services and the date 
the questionnaire was admin- 
istered* These conditions 
worked against detecting 
treatment-control di f fer- 
enceSf especially on the 
survey measures* 

A decision by the Deputy 
Superintendent for Field 
Services of the Chicago 
school 5 to del ete most 
self-report del inquency 
items from the student ques- 
tionnaire created a large 
problem for the evaluation 
of both Chicago projects* 
PCD was able to obtain 
del inquency data from an 
alternative source (police 
records)? but C3E refused to 
do so* 



These obsti:icle5 notwith- 
standing* the two Chicago 
projects were both part ial 1 y 
successful at i mpl ement ing 
exoer i mental desi gns* 

Inferences about the 
f ecti veness of the rema i n- 
i < projects on any outcome 
m res cannot yet be made* 
Wr tempted to evaluate the 
ef ti veness of the seven 
projects that ident i f ie J 
non-equi valent c om pa ris on 
groups by statistically 
adjusting for differences 
between the treatment and 
control groups that exi ste J 
before the treatment began* 
This activity proved to 6e 
of little value because we 
did not have pre-test meas- 
ures of the most relevant 
characteristics to use as 
St at i st i cal con trol s* Such 
adjustments are expected to 
be more useful as a fall- 
back design next year when 
good pre-test measures will 
be available in most cases? 
at least for many of the 
youths* 

Al though any of a number 
of projects coul d be used to 
illustrate the problem? the 
Const i tut ional R i ghts Foun- 
dation design orovides a 
good exampl e of tne f 1 aws 
inherent in this type of 
analysis* This project 
i nvol ves two treatments- -a 
Youth Committee and an 
Options Class* The selec- 
tion criteria for the two 
treatment s differed* The 
Youth Committee members were 
selected from a 30ol of stu- 
dents who had exhibited 
laadership; the Options 
Class members from a pool of 
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students who had experienced 
academic difficuTty* One 
would expect f given the 
selection procedurest that 
the Youth Committee oartici- 
pants woul d be cons i derabl y 
more i nterpersonal 1 y skilled 
than the average student in 
the school* The the Options 
Class participants would be 
expected to be lower academ- 
ically than the typical stu- 
dent in the school. This 
pattern of differences 
showed up when we compared 
the means on outcome meas- 
ures for the two treatment 
groups with each other and 
wi th the mean scores for a 
random sample of other stu- 
dents in the school (see raw 
meansf Table 4)« Only weak 
proxy measures of leadership 
abilityf prior academic dif- 
ficulty? and other group 
selection criteria were 
available* so we were unable 
to adjust the observed dif- 
ferences in the outcomes for 
the pre-existing differ- 
ences. The adjusted •'change 
in outcome** columns in Table 
^ reflect the difference 
between youth committee par- 
ticipants and the students 
in the random sample and 
between options class stu- 
dents and random sampl e stu- 
dents that remains after 
pr eex i st i ng di f f erences on 
family background measures* 
such as parents • educat i on 
levelf are taken into con- 
sideration. The adjusted 
mean scores for the groups 
still show what can most 
plausibly be interpreted as 
the pre-existing pattern of 
differences: The Youth Com- 
mittee is highest* the ran- 
dom samplfei is in the (riiddle* 
and the Options Cla»ss is at 
the bottom of the c^istribu- 
tion on most measur^e^s of 



social development* The 
group differences on outcome 
measures cannot be inter- 
preted as indicators of oro- 
gram effectiveness (or inef- 
fectiveness* in the case of 
the Opti ons Class) because 
the hyoothesis that the dif- 
ferences j^^flect pre-exist- 
ing differences has not been 
laid to rest. In the CRf 
case* we have additional 
evidence that the differ- 
ences may not have resultei 
from the program: The Youth 
Commi ttae intervention was 
not implemented as fully as 
was anticipated last year* 
and it is primarily intended 
to create structural rather 
than individual change* 

5li!Dir§iiye_Eyaiua t ion _ Design s 
l2£-l!2S«5econd_Proiect_Year^ 

The designs for making 
inferences about project 
ef f ec ti veness ( summa t i ve 
evaluation) for most proj- 
ects will be strengthened 
considerabl y in the second 
year of operation* An over- 
view of those designs is 
given in Table 5« Some 
designs are more difficult 
to implement than others* 
and some pr o j ec t sett i nqs 
are less amenable to experi- 
mental activities* The 
1 evel of re sources commi tted 
to evaluation activities has 
i n the past varied across 
p r o jec ts ; projects, may be 
expected to show disparate 
levels of effort* These 
issues al 1 influence the 
likelihood that planned 
evaluation designs will be 
i mp 1 emented* Such concerns 
contribute to the overa'll 
assessment of eval uatabi 1 i t y 
shown in the far right 
col umn of Table 5» 
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Three projec t s--Bronx* 
Charlestont and Peer Culture 
Development—stand out as 
highly evo?jatable projects. 
Three projects — Harlem? New 
Jersey? and St* Paut — are 
low in eval uatabi 1 i ty? at 
least in terms of summative 
evaluation* The remaining 
11 projects are evaluatable? 
but their designs are weak 
in some way? or there is 
uncertainty that the design 
will be impl emented as 
planned* 
r 

Summative evaluation is? 
of course? hot the only con- 
cern of an evaluation* In 
some cases where the designs 
to assess project effective- 
ness are rather weak? or 
even dismal? other "wiays of 
knowing" (Thar p C Gallinore? 
n*d*) can provide leads for 
more careful examination in 
a replication elsewhere or 
at another time* Because 
the time period for the 
Alternative Education Pro- 
gram is limited? however? 
the possibilities of pursu- 
ing these leads are also 
likely to be limited* This 
makes summative evaluation 
of these action projects a 
very important evaluation 
concern* 

PLQjg^ ^- g g sc r i p t i o n s 
Pro iec t _Size_and_Character- 

Tabi es 6 through 3 show 
the number of students 
served by each project and 
the ethnicity and socioeco- 
nomic level of these stu- 
dents as of April 1981 where 
this information is availa- 
ble from the SAES student 
questionnaire data collected 



in late Spring? 1981* 
School total enrollments are 
also shown* Some projects 
(e*g*« the Chicaga 3oard of 
Education's project) aim to 
provide intensive services 
to a smal 1 number of stu- 
dents* Others (e*g*? tha 
Kalama/oo project) provide 
only indirect services to . 
speci f ic students because 
they are primarily aimed at 
school structural changes* 

The ethnicity of students 
served varies across proj- 
ects* Some projects (e*g*? ^ 
Plymouth) serve predo.ni- 
nantly white studentr ies*<l> 
Others (e*g*? Harlem? Comp- 
ton? and Charleston) serve 
predominantly black popula- 
tions* Houston and Puerto 
Rico serve Span i sh-speak ing 
or Soani sh- sur named popul a- 
tionSf and the Bronx project 
serves a mixed Hispanic and 
black population* The Hay- 
ward project serves an Amer- 
ican Indian population* 

The level of parents* 
education is a measure of 
the rel ative social status 
of the clientele being 
served* Table shows that 
there is a considerable 
range of parental education 
across the projects* (The 
table entries show average 
parental educat i on? (mother 
♦ father)/2? where ••6*' means 
coTipleted four years of col- 
lege? and ••3*' means finished 
secondary school *) In 
Puerto Rico? for example? 
the average student's 
parents have not completed 
high school? whereas in 
Pasadena the average stu- 
dent's parents have gradu- 
ated from college* 
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Students receiving direct 
services in the Alternative 
Education Program are 
remarkably similar in eth- 
nicity and family educa-* 
tional background to other 
students in the same 
schools* Isolation of 
minority or low socioeco- 
nomic status students does 
not generally appear to be 
occur r i ng« 

lSJt)22l-§Q^. Commu ni t^ ontext 

Tables 9 through 11 
describe the community and 
school contexts? student 
compositionv and the charac- 
teristic school governance 
for the 53 schools from the 
15 projects included in the 
first annual survey adminis- 
tration (Miami and Milwaukee 
were not included)* All 
characteristics reported in 
these tables are measured by 
scales constructed from 
items in the student ques- 
tionnaire* See Chapter 5 of 
this report f or descr ipti ons 
of the scales* 

These tables are 
abstracted from more 
detailed profiles based on 
both teacher and student 
surveys* wh i ch were d i str i - 
buted to project administra- 
tors at the end of the first 
program year* The charac- 
teristics presented here are 
based on student survey 
scales only* They are 
intended to provide a char- 
acterization of gross proj- 
ect- to-project differences 
in the co/nmun i t i es and stu- 
dents servedf as wel 1 as a 
description of the school 
env i ronment s * 



Each row in the tables 
summarizes a school's place- 
ment in the distribution of 
mean scores for all 60 
schools* A indicates 
that the aver ac]e score of 
all student reports in th-? 
school was above the 75th 
percentile for all schools* 
A indicates that the 

average fell belo>« the 25th 
percentile* A indicates 
that i terns used in the scale 
were not measured at the 
school * 

The tables indicate that 
the projects wi th al terna- 
tive schools that are sepa- 
rate from the school sys- 
t em-- Houston ♦ Compton ? and 
Lac Courte Oreilles--have 
the most positive c 1 i mates * 
Students from all three of 
these projects report that 
the rules in their schools 
are fair* Students in Comp- 
ton and LCG r eport h i gh lev- 
el s of s tuden t- te':icher 
inter acti on? planning? and 
clarity of rjles* Students 
in Houston and Comoton are 
less alienated and more 
attached to school f and stu- 
dents in Ho us ton and LC3 
report lo»^er levels of 
school cJi sruption than do 
students in mos t o ther 
school s . These three pro j - 
ec t s are among the four w i th 
the highest levels of del i n- 
quenc y * 

Anothe r general pattern 
whi ch emerges from Tabl es 9 
through ll is that the 
Puerto Rico project operates 
in an exceptionally pleasant 
cl i mate: tow 1 evel s of di s- 
ruption? deli nquenc y* and 
a I i enat i on and high levels 
of attachment? bel ief f and 
each of the five governance 
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character i St ics are 
reported. Charleston and 
Kalamazoo also operate in 
fairly positive settings* 
although not nearly as posi- 
tive as Puerto Rico* 

The Chicago projects* on 
the other handt are oper^it- 
inq in less than ideal con- 
ditions* One or the other 
of the Chicago projects is 
extremely low on four of the 
f i ve governance character i s- 
tics* and students in most 
of the RETAIN schools report 
high levels of community 
crime and al ienation* 
Interestingly* the PCD 
schools report low levels of 
disruption* The New Jersey 
and Virgin Islands projects 
also have extreme negative 
scores on many d i mens ions* 
although Virgin Islands* 
despite high community crime 
and low attachment and 
belief* scores at the low 
end of the del inquency dis- 
tr ibut ion* 

Finally* it is interest- 
ing that students in both 
New York City projects (PREP 
and Jazzmobile) are among 
the three lowest-scoring 
projects on belief in the 
val idity of conventional 
rules? ana that students in 
projects located in the nore 
af f I uent Kal amazoo score 
extrerriely low on attachment 
to school* 

The program models for 
the 17 alternative education 
projects are very different* 
Although all projects incor- 
porate at least one of the 
desired project models spe- 
cified in the RFP* the 
degree of overlap between 



trte solicitation require- 
ments and the actual program 
models varies considerably* 
The Charleston project* 
PaTHE* includes almost ev^ry 
program element described in 
tne RFP* The Peer Culture 
Devel opment project * on the 
other hand* focuses most of 
its resources on only one of 
the ni ne program ^odel s — al - 
tering peer group experi- 
ences* 

In short* the Al ternative 
Education Program includes a 
broad range of projects* 
Some are attempting to 
implement a wide variet/ of 
i nter vent ions * and others * 
only a few kinds* Projects 
impl ement ing onl y ?i few 
i nter vent i on s appear* in 
gener aV * to be f ocu s i ng iios t 
of their resources on those 
interventions and may there- 
fore be implementing them 
with considerable strength. 
By doing sOf however* they 
may be neglecting many addi- 
tional ooport uni t i es for 
intervention* Those proj- 
ects implementing a variety 
of kinds of intervention are 
hittinq the system in many 
places at once* but run the 
risk of diluting the 
strength of specif ic int^^r- 
ventions by overextend inq 
themselves. 

Four general types of 
pi anned intervention catego- 
r i es may be i dent i f i ed* and 
the f oil o wi ng subsec t i ons 
illustrate interventions in 
each category* 

One type of i nter vent ion 
aims to alter the content or 
method of instruction in the 
schools in an attempt to 
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improve youths^ motivation 
and coHimitment to education 
as well as their chances of 
success in school* Two main 
strategies are used: Proj- 
ects attempt to individual- 
ize instruction either 
through tutoring by teach- 
ers? peers? or outside 
agents? or through the 
development of individual- 
ized learning plans* The 
Charleston? Chicago Board of 
Education? and Compton proj- 
ects use both strategies* 
Kalamazoo uses tutoring? and 
the 3ronx project uses indi- 
vidualized learning plans* 

Another strategy for 
altering the delivery of 
academic instruction is cur- 
r icu 1 um de vel opment • Some 
projects focus on altering 
the content of the curricu- 
lum* Jazzmobile? for exam- 
ple? offers courses in the 
arts? and Project STATUS in 
Pasadena offers Options 
Classes focusing on English 
and social studies? and has 
initiated a Leadership 
Training Class* Compton and 
the Jewish Vocational Ser- 
vices project in Milwaukee 
emphasize practical voca- 
tional skills development in 
their courses* The Bronx 
and Houston projects offer 
instruction in cultural her- 
itage* Curriculum develop- 
ment also comes in the form 
of methods development* 
Project STATUS uses small 
groups and simulations in 
its classrooms; Charleston, 
uses Student Team Learning; 
and the Chicago Board of 
Education? Lac Courte 
Oreilles (Hayward)? and 
Compton use computer-as- 
sisted instruction* 



Reward systems are the 
focus of some projects* 
interventions* The Miami 
project model calls for the 
establishment of .1 token 
economy system. Project 
PR£P in the aronx tries to 
implement a ••time-out" room 
for students who are falling 
behind academically or dis- 
rupting the classroom* 
Non-traditional re- 
wards — such as field trips? 
sweat Shi rts? opportunities 
to display artwork in thn» 
communi ty--ar e used in the 
Bronx? the Virgin Islands? 
Pasadena? Harlem? ind Hou- 
ston* 

Finally? the St* ^aul 
oroject is implementing a 
structure to al ter the 
nature or process of learn- 
inq* It uses Action Learn- 
ing Projects wherein stu- 
dents design and execute 
projects to help solve a 
school or community problem. 
Taachers serve as managers 
of learning rather than 
ins tr uctor s * 

l§L!ti£§«^£iLy.SL^ • * sec- 
ond major program component 
focuses on improving the 
schools! delivery of ser- 
vicesa This is often accom- 
plished through counseling: 
Peer Culture Development? 
St* Paul and Charleston use 
peer counseling? and Ply- 
mouth and the Bronx use 
individual counseling* Mil- 
waukee* Puerto Rico? Hou- 
ston? and Miami focus on 
vocational counsel ing« 
Pasadena? a project cov;* ^e- 
1 or with a caseload of about 
120 students provides ser- 
vices to both students and 
parents* 
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Various classroom inanai<je- 
ment techniques are also 
used in attempts to meet the 
needs of students: Virgin 
Islands and Kalamazoo imple- 
ment some kind of a ••family 
group" period. During this 
period? snail groups of stu- 
dents and a teacher discuss 
problems the students are 
experiencing. In Miami? 
classrooms are managed using 
principles of applied beha- 
vioral analysiSf and in 
Compton and the Virgin 
Islands teachers are 
selected and trained to 
avoid negative reinforce- 
ment. In Pasadenat students 
in the var ious pro ject com-- 
ponents formulate their own 
rules for classroom manage- 
ment. 

Student participation in 
decision making is another 
affective strategy* 
Charleston? Kalamazoot and 
Project STATUS involve stu- 
dent groups in school -level 
decision making. The Puerto 
Rico project also uses stu- 
dent involvement? but the 
focus is on involving stu- 
dents in making decisions 
about their own education. 
A number of projects attempt 
to increase students' 
i nvol vament in extra-curri- 
cular activities either by 
starting up new activities 
and clubs (Puerto Ricot 
Charleston) f by providing 
opportunities for students 
to go on trips (Charleston? 
Bronx)t or by organizing 
sports teams (Houston). 

different strategy aimed at 
meeting students' affective 
needs might be called gen- 
eral school climate improve- 
ment. For example? the 



Charleston project focuses 
on cl imate improvement by 
organizing a "school pride 
campaign" aimed at improving 
teacher? student and commu- 
nity percept i on s of the 
school through tha use 
the mediat pep rallies? cle- 
an-up campaigns? etc* The 
project also sponsors 
faculty team-building activ- 
ities to improve morale. 
Ort for example? the 
Pasadena project lims to 
influence school cl imate 
through its Youth Committee. 

The St. Paul project? 
like the Charleston and Vir- 
gin Island projects^ focuses 
heavily on public relations 
activities^ It has been 
active in helping schools 
explain to their constituen- 
cies what it is trying to 
do. It has worked wi th 
school staffs to get stu- 
dents involved in writing 
for community newspapers an 
providing pictures for those 
papers. A group of students 
has organized a iiemorial 
fund for an outstanding 
teacher? and shjred in publ- 
icity efforts. 

Altering overall school 
organization and management 
is another school organiza- 
tion intervention* The 
Kalamazoo project is exem- 
plary in this area. Its 
program model* calls for 
establishing and maintaining 
an organizational structure 
composed of area task 
forces? a building steering 
committee? and an advisory 
council that engages stu- 
dents ? teachers ? bu i 1 di ng 
admi ni str^tor s? di str ict- 
1 evel school admi n i s tr ator s ? 
parents? and juvenile jus- 



-1^9 



ERIC 



169 



Qverv ieM 



tice service administrators 
in joint decision making and 
in the implementation of all 
aspects of the projects 

Charleston is a project 
which alters the management 
of the school in more spe- 
cific ways: A di sc ipl in^^ry 
referral procedure aimed at 
increasing the consistency 
of rule enforcement is 
established? and business- 
education partnerships 
between community business 
representatives and building 
principals are formed* 
These ptirtner sh i ps offer 
expert management assistance 
to school administrators. 
Other projects? including 
the Puerto R ico and St. Paul 

P^'OjectSf ^Vsp ,3xe wo'''^^ 
on developing partnerships 
with community businesses* 

The St» Paul project aims 
to alter school organization 
and the way the schools are 
perceived by the public by 
i ncreas i n j school -communi t y 
linkages* It also seeks to 
improve school governance by 
proviaing training for 
teachersf working witn 
administrators to foster 
improvements in school rules 
and their admi n i str at i ont 
and getting students 
involved in working with 
teachers on the development 
of these rules* In addi- 
tiont it is developing an 
adv i sor-adv i see^sys tern in 
its school s* This system i s 
intended to provide small 
groups for students to dis- 
cuss pr obi ems and obta i n 
i nformat iont and to receive 
more individual attention 
than woul d other wi se be pos- 
sible* It is also working 
to create more active stu- 
dent councils and has 



created a school jdvisory 
council with parents? stu- 
dents and staff represented* 

C ommu Qi t^^iQ vg 1^ ve ment * 
The fourth major program 
component i s community 
i nvol vement* Communi ty 
i nvol vement i s encouraged 
b^th for the purpose of 
channel i ng communi ty 
resources into the schools 
and for hel ping individual 
students or cl ients* 
Parents are the primary com- 
munity contacts* Charles- 
ton? for example? involves 
parents in school improve- 
men t act i v 1 1 i es * 3t her 
projects (e*g*f Compton? 
Chicago Board of Education? 
Puerto RicOf Bronx) seek to 
increase parental involve^ 
ment in their own children's 
educations by informing them 
of students' progress and of 
school activities* And? the 
Pasadena project nakes use 
of an Advisory Action Com- 
mittee and the parents of 
students participating in 
its project components as 
resources for field trips 
and internships* Finally? 
business contacts are used 
as resource s to improve 
schools (e*g*» St. Paul) and 
to employ youths (Milwau- 
kee ) * 

The foregoing classifica- 
tion is intended to provide 
the reader with a general 
overview of the types of 
or ogr ams funded under the 
al ternat i ve educati on i ni t i - 
ative* It is based for the 
most part on project plains* 
Detailed descriptions of the 
program plans and of what 
was actual 1 y impl e men ted at 
each project site during the 
first year of operation are 
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found in the descriptions of 
individual pro jects. 

Institutional Ch ang e 

□JJOP stressed an organi- 
zational focus and an organ- 
izational change approach in 
its request for proposals 
for the initiative* and 
repeated these themes conti- 
nually during the first pro- 
gram year. The RFP called 
for programs that seek to 
reduce delinquency by 
••changing the structure and 
the educational processes of 
school in such a way that 
students would maintain or 
develop a stake in academic 
achievement and conformity 
to conventional rules? and 
woul d be 1 ess 1 ikWVf to ^ - 
become alienated and engage 
in delinquent activities* 

One can imagine alterna- 
tive organizations of the 
American educational enter- 
prise* Compulsory education 
as it exists today might be 
abolished in favor of work- 
study relationships between 
adolescents and an employer* 
Schools might exist solely 
as resource centers* Ort 
they could supplement a tra- 
ditional curriculum with 
on-the-job training compo- 
nents* Large schools could 
be broken up into clus- 
ters — school s wi thin 
school s* Less drastic 
organizational changes might 
include altering grading 
practices so that students 
do not experience academic 
failure in school t or giving 
student coalitions the 
responsibility for develop- 
ing and maintaining school 
discipline codes* The pos- 



sibilities are nearly 
endl ess* 

Considering this wide 
range of possibilities? the 
or o jects funded in the 
al ternative education initi- 
ative focus minimally on 
organizational change* 
Insteadf the projects gener- 
ally stress an alternative 
approach to dealing with a 
troublesome subpopul ation of 
youths* Some do? in addi- 
tion? direct their efforts 
toward influencing specific 
school policies and proce- 
dures? and some attempt to 
augment the existing school 
curricula with materials or 
methods having wider appeal* 
Some create organizations 
outside the public school 
sys tem but pr^pvide schooling 
that resembles traditional 
education in a modified set- 
ting and with somewhat dif- 
ferent emphases* But none 
of the projects concentrates 
on making changes in the 
basic educational structure 
of the kind that would radi- 
cally change the condition 
of youth in contemporary 
SDC i ety (cf * Gottf r edson* 
1931 ; Greenoergt 1^77 ; Pre- 
sident's Science Advisory 
Committee on Youtht 1967)* 

The projects might be 
ranked on criteria related 
to organizational change 
focus f such as the percen- 
tage of resources expended 
on attempts to change exist- 
ing structures* practices? 
or policies* or the percen- 
tage of critical oenchmirks 
met related to organiza- 
tional change* Such a rank- 
ing would be misleading 
because it fails to consider 
the history of the organiza- 
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tions and the organizational 
settings in which the proj- 
ects operate* The Charles- 
tonf Kalamazoov and St» Paul 
projects — all school system 
projects— i^oul d rate high* 
while the ^oustont Hdrlem« 
and Miami projects — all com- 
munity-oased organization 
projects— woul d rate low on 
a measure of staff time 
devoted to changing existing 
public school practices* 
This ranking ignores the 
prior work done by the CSO^s 
in developing an alternative 
or addition to the existing 
System* The proof of the 
pudding might be in the 
degree to which the various 
alternatives are adopted by 
or replace the system to 
which they provi de an al ter- 
nat i ve« 

Systems are not easily 
Changed* The environment 
must be ready to accept 
change; i*a«« there must be 
dissatisfaction with the 
status quo» An alternative 
model for organization and 
management of the system 
must be present? as well as 
a planned process for manag- 
ing the change* Charismatic 
and dynamic leadership also 
facilitates change* 
Final lyt the cost of the 
proposed change must De less 
than or equal to the cost of 
maintaining the present sys- 
tem* These criteria were 
incorporated into a list of 
indicators of the 1 ikel ihood 
of school change* by Noel 
Day of Polaris (the Techni- 
cal Assistance Contractor 
for this initiative)* Table 
12 is adapted from the cri- 
teria used in Day's ratingt 
and Figure 1 presents the 
ranking* Oay did not rate 



several projects 
(Charleston* Puerto Rico* 
Miami* New Jersey) with 
which he was unfamiliar* We 
have placed these projects 
into the figjre where they 
appeared to bel ong* using 
iiterpol aticn based on 
information developed by our 
evaluation* On the wholet 
the assessments made oy Oay 
jibe well with wh^t we know 
about the projects* We 
adjusted Day's rankings for 
tnree proj ec t s — Compton* 
Pasadena* and Virgin 
Islands--on the basis of new 
i nf ormat i on about these 
projects • 

These rankings* of 
course* are an interpreta- 
tion of hunches about the 
likelihood of creating 
change in existing school 
systems* Although we under- 
stand OjJCP's desiro to 
create such change* and to 
sponsor demonstration proj- 
ects likely to create it* 
two cons i dera t iona make js 
douDt the utility of an 
excess ive focus on insti tu- 
t i onal change in school sys- 
tems at present * 

First* one widely hel d* 
1 egi t imate perspec ti ve on 
the public schools is that 
the public schools would 
requ i re such extens i ve 
restructuring to serve fill 
students wal 1 thit* rather 
than seeking to change themt 
we shoul d be seeking alter- 
natives to them* It has not 
escaped our not ice that 
those projec t s to the far 
left (no pun intended) in 
Figure 1 are creating alter- 
natives to the publ ic 
schools. 
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Secondf it is at present 
preir.ature to assess the 
desirability of institution- 
alizing any of these proj- 
ects. They have not yet 
demonstrated effectiveness 
at reaching their stated 
goals and objectives. 
Despite the apparent enthu- 
siasm on the part of some 
education departments for 
replicating or extending 
action project activities 
elsewheret these projects 
are of uncertain effective- 



ness, rte are convinced that 
assessing project effec- 
tiveness is at present more 
important than focusing on 
extension or replication. 
Widespread public percep- 
tions of the ineffectiveness 
of social programs are pro- 
bably due* at least in part* 
to premature efforts to 
implement new programs every- 
where at once without thor- 
ough and systematic €es_ting 
and development. 
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Footnotes 



I. These tables may not* however, convey the entire pic- 
ture. For example, according to the St. Paul ^^^^rterly 
report to OJJOP for the fourth .quarter of 1982, 36t of the 
cummulative number of students receiving direct.seryices 
from the project were Black. This contrasts with the pro- 
portion of Black students in the schools estimated from the 
student survey. Unfortunately, these percentaqes can not be 
estimated dependably using questionnaire data for this proj- 
ect. A number of conditions, includinq differential mobil- 
ity of students of different ethnic groups and differential 
non-response for different ethnic groups, sometimes makes 
the racial composition of a school's studentry difficult to 
estimate from survey data. 
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Figure 1 

Rankinq of Alternative Education Projects on Likeli 
Institutionalization in School System 



ho^d of 



JVS--Mi 1 waukee Puerto Rico 



Ja/Zftiobi le-- 
Har lem 

Hayward 

Houston 



Compton 
New Jersey 



CRF--Pasaden3 
Miami 

PCO--Chicago 
Kal amazoo 



RETAIN — Chicago 
P lymouth 
PREP--8roTix 

St. Paul 

Virgin Islands 
PATH6--Charleston 



LESS LIKELY 



MORE LIKELY 



Note. Adapted from a figure by Day (I931)* The Charlestont 
Puerto P.icof Miami f and New Jersey projects were omitted from 
Mr# Oay*s ranking* The evaluation staff produced rankings for 
these projects and modified Day's rankings for three other 
projects--Compton» Pasadenat and Virgin I si 3nds--Dased on addi- 
tional i nf or mat i on« 
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Table 1 



Overview of Implementation Status of the Al terridti ve 
Education Projects: August 19ftl 



Pro ject 
Compton 



Implementation Status 



Is providing some of the anticipated ser- 
vices to students. Has had high staff and 
student turnover and difficulty implementing 
some project components. 



CRff Pasadena 



PCDf Chicago 



C3Et Chicago 



Kal amazoo 



Bronx 



Jazzmob i 1 et 
Marl em 



Is implementing some of its anticipated 
project components? but not others; used the 
Sprint as start-up time to write experimen- 
tal curriculum. Has had difficulties attri- 
butable in part to a lack of careful proac- 
tive project planning? and in part to 
limited school system commitment to the 
project. 

Is replicating some well practiced proce- 
dures in nearly the form intended. Its 
range of interventions is fairly narrow? but 
they appear to be implemented with consider- 
able strength. 

Is providing some of the intended services? 
although not with the strength or integrity 
initially hoped for. Experienced difficul- 
ties starting up? and has not oeen charac- 
terized by systematic proactive planning. 

Is implementing part of the activities 
intended. Experienced difficulties due in 
part to changes in school administration not 
foreseen when the project was planned. In 
addition? the project director does not have 
control over sufficient personnel resources 
to implement all components of the program 
fully. 

Is providing many of the anticipated ser- 
vices to students? although the strength of 
these is unknown at present. Experienced 
turnover at the project-administration 
1 evel . 

Is providing arts instructiont although the 
exact nature of these activities remains 
unclear. 
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Tabi e 1 (cont • ) 



Project 
Charleston 



Impl e men tat i on Status 



Is i mpl ewent i nq 3 wide range of interven- 
tions; some are not implemented in the form 
or with the strength intended* ^ay be ove- 
rextending itself in attempts to Jo too 
much* 



Houston 



Virgin Islands 

LCOf Hayward 

Puerto R ico 

M i ami¥ 
P 1 ymouth* 

New Jersey^^ 

M i 1 waukee* 

St^ Paul^ 



Is continuinq the relatively smooth opera- 
t ion of an a1 tern at ive school already in 
existencet but has not extended activities 
much • 

Is implementing one of its major interven- 
tion St but has delayed the implementation of 
others. 

Has installed compute r terminals? but is 
implementing little beyond this limited 
i nterventi on. 

Is implementing mo>c planned interventions* 
and is changing some components to 
strengthen them. 

Impl emented a si x- week summer p i 1 ot test. 

Implemented some direct services to stu^ 
dents. 

Did not begin project implementation in the 
past year. 

^eqan implementation of direct services near 
the end of the past year. 

Implemented several project components. The 
acting project director's appoi ntnent as 
project di rector was delayed* slowing impl e- 
mentF tion. 



*These projects 
others. 



were funded four to five months after the 
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Table 2 

Student Questionnaire Response Rates: Spring 1981 



Project Name 


School Code 


M in Sample 


Response Rate 


PI ymouth | 


31 1 

41 i 

42 1 

43 1 


84 
85 
111 
94 


.87 

1 .87 

1 .66 

1 


Kalamazoo f 


316 1 
327 1 


298 
278 


1 .76 
1 .80 


Charleston | 


242 1 

741 1 

742 1 

743 1 
751 1 
754 

755 
944 
951 


323 
350 

338 

3' ^ 

J o 

1 310 
1 386 


1 .96 
1 •95 
1 •''^ 
1 .83 
1 • 84 
1 .84 
1 .89 
1 .96 
1 .59 


Compton 1 


101 


1 92 


f •39-1.00<t>> 


Jazzmobile | 


88 


1 819 


1 .33 


Virgin Islands| 


1101 


1 295 


1 .83 


Lac Courte 1 
Oreilles | 


1201 


1 125 


1 -55 


Chicago I 


6130 

5880 

5750 

5090 

1240 

1340 

1430<a> 

2 300 

4440 

4550 


I 233 
1 2 47 
1 263 
1 161 
1 376 
1 367 

1 163 
t 131 
1 235 


1 .36 
1 . 88 
1 .37 
1 .14 
1 . 46 

1 

( .91 
1 .96 
1 .49 



a. See Peer Culture Development 

b. Information from different project staff memoers apout 
the school enrollment at the time of the questionnaire 
does not converge. 
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Table 2 (cont«) 



Project Name 


School Code 


N in Sample 


Response Rate 


Constitutional | 


70 1 


291 1 


.79 


Rights 1 


82 1 


324 1 


.58 


Poundation ; 


1 






I 

Peer Culture f 


1 

1370 1 


334 1 


.65 






J DO 1 


• 5 7 




1820 1 


360 1 


.79 


1 

St* Paul 1 


1 

210 1 


306 1 


.37 






Tknci 1 
^ uU I 


• J iJ 


1 


230 1 


302 1 


.66 


t 


3^2 1 


324 1 


.39 


1 


352 1 


302 1 


.56 


1 

Puerto Rico | 


1 

301 1 


556 1 


.90 


1 


302 1 


2 30 1 


. 83 


1 


603 1 


596 1 


.67 


1 

9ronx | 


1 

22 1 


296 1 


.70 


1 


55 1 


183 1 


.81 


1 


63 1 


152 \ 


.64 


• 


6^ 1 


139 1 


.79 




Oc 1 


309 1 


. 7h 


j 


117 1 


333 1 


.71 


1 


132 1 


156 r 


.69 




• V5 1 


301 1 


.34 


1 


\ >7 ' 1 


311 1 


.39 




i Ha 1 


C 97 1 


.61 


j 


166 1 


2 64 1 


.61 




229 \ 


355 1 


.77 


New Jersey | 


001 1 


820 1 


.59<c> 




002 1 


299 1 


.69 




C03 1 


375 1 


.40 




004 1 


379 1 


.65 


Houston f 


1001 1 


B4 1 


.97 



c. Response rate for this school estimated ^no roster available) 
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Table 3 

Summary of Experimental Designs 
for First Program Year 



Project 



Design allows for comparT^n of program and noji^progr^ 
S tudentsj 



Compton 

CRF, Pasadena 

PCD, Chicago 

CBE, Chicago 

Kalamazoo 

Bronx 

Jazzmobile, Harlem 
Charleston 
Houston 
Virgin Islands 
LCO, Heyward 
Miami 
Plymouth 
New Jersey 
Milwaukee 
St. Paul 
Puerto Rico 
i7 



equivalent nonequivalent 



no 
no 
yes 

c 

yes 
__b 

no 
no 
no 
no 
no 
no 



no 



a 
a 
a 



no 



no 

yes 

yes 

yes 
__b 

yes 

yes 

yes 

no 

yes 

yes 
a 

yes 
a 

a 

a 

yes 



Schools 



(all noneqgivali^nt) 



no 
no 

yes 

yes 

yes 

no 

no 

yes 

no 

no 

no , 

no 

no 

no 

no 

no 

no 



b) 



"^ect services to students did not begin or began on a partial basis 
during the first program year. 

This project, by design, did not offer direct services to youths during 
the first program year. 



Design deteriorated on implementation. 
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Table A 



Details of Analysis of Effects with Protected 
Tests of Significance for Project STATUS 





Raw Mean 




Adjusted 


change 




Youth 


All 


in outcome 


due to: 


Criterion Variable 


Cmte. Options 


Others 


Y. Cmte. 


Options 


Muir 


Self-Reported grades 


3.12 2.56 


2.65 


+.42* 


-.50 


Involvement 


2.05 .01 


-.38 


+2.24* 


a 


Elliot 


School nonattendance 


2.74 1.81 ; 




+.81* 


a 


Practical knowledge 


13.01 11.56 


12.47 


+ .58 


-.90 



Since the Options Group was correlated less than 0,10 w/ the criterion, it was 
not included in the analysis. 



Significantly different from adjusted mean for all others, p<,,05. 
Significantly different from the mean for all others, p<.01. 
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Table 5 



Project: 



Compton 



CRF, 

Pasadena 



PCD, 
Chicago 



CBE, 
Chicago 



Evaluatability Summary 
for Second Program Year 



Brie^ Description 



Random assignment of new referrals 
to program and control. Most pro- 
gram slots are filled by returning 
students. Pre-treatment school and 
police records for both groups. 
Extremely small sample size works 
against detecting differences. 

Non-equivalent comparison group com- 
posed of volunteers, referrals, etc., 
who did not enter program. SAES pre- 
test on all. 

Random assignment to treatment and 
control for all students. Large 
sample size. Court and self- 
reported delinquency data available. 

Random assignment to treatment and 
control for all students. Randomiza- 
tion process not carefully monitored. 



Strength 
of 

Design 



Likely 

Integrity 

of 

Design 



Overall 
Evaluat- 
ability 
Rating 



2- 



3+ 



3- 



Kalamazoo 



Bronx 



Ja?;zmobile, 
Harlem 



Most change is expected to occur at 2 
the school level rather than to 
individuals being served. The 
design will allow for comparison 
of change from year one to year two 
of the program school with a control 
school using the SAES questionnaire. 
Comparisen of students directly 
served and not served will be weak. 

Random assignment to treatment and con- 4 
trol. SAES pretest on both groups. 

Project has no control over assignment, 1 
but school's assignment is thought to 
result in nearly equivalent groups 
of treatment and nontreatment youths. 
Likelihood of measuring outcomes is 
extremely low. 



0 
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Table 5 Continued 



Project 



Charleston 



Houston 



Brief Description 



Strength 

of 
Design 



Virgin Islands 



LCO, 
Hayward 



Random assignment to treatment and 3+ 
control in all seven schools to 
assess *individual-leve]^ and two 
comparison schools to assess organ- 
izational level^ program effectiveness. 
Delinquency measures limited to 
official records in middle schools, 
but pre and post treatment official 
records available. Large sample 
size . 

Design allows for comparison of stu- 2 
dents in two program components (non- 
equivalent groups) and a small non- 
equivalent no-treatment group. Pre- 
treatment data from records available 
for all groups . 

Regression discontinuity design for ^ 
one project component (non-equiv- 
alent groups with SAES pre-test 
as a backup). Small sample size. 

Evaluation of PLATO component only. 2- 
l Nonequivalent groups with SAES pre- 
test on relevant outcomes. 



Likely 
Integrity 

of 
Design 



Overall 
Evaluat- 
ability 
Rating 



Miami 



Comparisons of nonequivalent treat- 
ment and control groups. Pre-treat- 
ment data from records for both groups, 
and pre-treatment skill test data 
for both groups. 



Plymouth Random assignment to program and con- 2 2 

trol with SAES pretest for the two 
high school components. 

Nonequivalent comparison groups with 
SAES pretest for middle school com- 
ponent. No comparison group for out- 
of -school (Growthworks) component 
because program serves all problem 
youths. Growthworks component not 
evaluatable . 
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Table 5 Continued 



Project 



Brief Description 



New Jersey 



Milwaukee 



St. Paul 



Puerto Rico 



Strenp^C.h 

of 
Design 



Non-equivalent comparison groups of 
students served and not served. Pre- 
treatment data from records. 

Random assignment to treatment and con-^ 
trol groups with SAES pretest for 
both broups. Multiple follow-up 
using SAES survey and data from offi- 
cial records. 

Most change is expected to occur at 
the school level rather than at the 
individual level. Comparison of stu- 
dents receiving and not receiving 
service will be extremely weak. De- 
sign allows for comparison of program 
and nonprogram schools on a non-SAES 
questionnaire. No relevant compar- 
isons on any measure of delinquency. 
Schools undergoing concurrent admin- 
istrative reorganization. 

No experimental control over program 
access; but small preexisting differ- 
ences between service recipients and 
other students, large-sample 
multi-wave questionnaire and official 
delinquency data allow for a non- 
equivalent control group design. 



1 + 



2+ 



Likely 
Integrity 

of 
Design 



Overall 
Evaluate 
ability 
Rating 



0 



Note: The rating schemes used for strength of the designs are as follows: 

random assignment of subjects to treatment and control conditions with pre- 
treatment measures on the relevant outcomes is given a code of '^4. 
Random assignment without pre-treatment data gets a code of ''3.** 
tdentification of a nonequivalent comparison group (or school) with pretreat 
ment information is coded as "2," and nonequivalent comparison groups without 
prptreatment data is coded as "1.'* Pluses appended to these codes indicate 
strong points (such as large sample sizes or multi-level designs) and minuses 
indicXLe weaknesses such as extremely small sample sizes, incomplete or non- 
existent data on delinquency, or flawed pretesting conditions. 

Integrity of design is a three-category forecast. A »»2" means that faithful 
implementation is highly likely; a "1^^ means that faithful implementation is 
problematical, and a ''0'' means that, faithful implementation appears unlikely. 



ERIC 



-165- 



1S5 



Table 6 

Total Enrollment and Number of Students Receiving Direct Services 

as of April, 1981 



Project 1 



School 



Total 
Enrollment^ 



Compton 

Constitutional 
Rights Foundation 

Peer Culture 
Development 



Chicago Board of 
Education 



Alternative School 

Muir S. H. 
Elliot J. H. 

Lakeview 

Curie 

Harrison 

Spry 

Pope 

Edwards 

Hearst 

Nettlehorst 

Ravenssrood 

Lemoyne 
Bontemps 
Blaine 
Gage Park 
Bowen High 
Lakeview High 
Thorpe 
Nightingale 
Sheridan 



Number of 
students Served^ 



61 

2120 
1325 

1366 

3065 

1100 
__d 
__d 

^-d 
„d 
__d 
__d 

515 
756 
718 
1400 
2700 

1366 
933b 
736b 

2114b 



92a 

111 
121 

89 
87 
82 
26 
32 
16 
28 
29i 
34 

14 
15 
15 
15 
14 
14 
15 
15 
15 



Kalamazob 
■* 

Bronx 



Mllwood 



CJHS 


//22 


CES 


55 


CES 


63 


CES 


64 


CHHS 


82 


CJHS 


117 


CES 


132 


CHHS 


145 


CIS 


147 


CIS 


148 


CIS 


166 


CIS 


229 



657 



1003, 
717 
757 

1065; 
824^ 

10 7 r 
673^ 

III9J 

1499? 
861° 
225^ 
589'' 



9 
15 
15 
16 
17 
14 
18 

6 
18 
17 

9 

1 



ISB 
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Project 



I 



School 



Total 
Entollment 



Number of 
Students Served 



Jazzmobile 



Puerto Rico 



Charleston 



Houston 

Virgin Islands 

Lac Courte Orellles 

Plymouth 



St. Paul 



Intermediate School 
Elementary School "A" 
Elementary School "B" 

Santiago Gonzalez 

Ruiz Belvis 

Dr. Alfredo M. Aguago 

C.A. Brown H.S. 
Burke H.S. 
St. John's H.S. 
Courtenay M.S. 
A.B. Rhett M.S. 
Rivers M.S. 
Haut Gap M.S. 

G. I. Sanchez 
Alternative School 

Elena Christian J. H. 

LCO Alternative School 

East Middle 
Central Middle 
Canton High 
Salem High 

Murray J.H. 
Washington J.H. 
Johnson J.H. 
Como S . H . 
Central S. H. 



506 

504 
446 

554 
632 

1059 

786 
1017 
798 
525 
476 
545 
450 

84 



1393 
92 

860 

924 
2350 
2387 

532 

702 
1407 
1094 
1101 



364 
146 
134 

82 

39 
120 

96 
98 
89 
99 
97 
92 
97 

84 



56 

c 

70 

25 
25 
49 
32 

32 
0 
0 
0 
0 



1 No direct program services began during the first program year in the Miami, N.w 

Jersey and Milwaukee projects, 

2 This figure comes from the principal questionnaire, unless otherwise indicated. 

3 This figure is the number of students who were reported to be receiving direct 

program services at the time of the annual students questionnaxre , unless 
otherwise indicated. Some projects such as Kalamazoo, St. Paul, and Charles 
ton have school change as a major focus, and serve students indirectly, 
a This is the number of students ever served. The school has a transient population. 

b These enrollment figures are taken from school rosters or reports from the districts' 
Evaluation and Research Office. 

^ In addition, the project served 33 students at- the Youth Center who were not 
enrolled In the alternative school. 

d Only program participants surveyed. Total enrollment unknown. 
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Table 7 



Ethnic Self- Identification of Students Receiving Services 
and the General Studentry In Project Schools 





Native 


Asian 


Span. 










jrrojecL ana group 


Am 

Am. 


Am. 


Am. 


Black 


White 


Other 


N 


Compton 
















Receiving svcs. 
Gen'l studentry ^ 


0.0 


u . u 


-1 o / 

19 • 4 


80.6 


0.0 


0.0 


36 
— 








— * 


— 


— 


Pasadena 
















Receiving svcs. 


0.5 


3, 7 


15.9 


42.9 


31.7 


5.3 


189 


Gen'l studentry 


0.5 


4.0 


17.9 


42.8 


30.3 


4.5 


201 


Peer Culture Development 














Receiving svcs. 


1.7 


1.0 


36.8 


36.5 


21.3 


2.7 


296 


Gen'l studentry 


0.6 


2.2 


44.0 


23.2 


28.3 


1.8 


505 


Chicago Bd. of Educ. 
















Receiving svcs. 


3.3 


0 . 7 


35.5 


43.4 


13.2 


3.9 


152 


Gen'l studentry 


1.9 


2.5 


36. 7 


44.6 


12.6 


1.7 


1113 


Kalamazoo 














Receiving svcs. 











— 


— 


— 


Gen'l studentry 


0.9 


1.4 


3.5 


28.7 


62.3 


3.2 


432 


Bronx 
















Receiving svcs. 


1.1 


I . L 


J5 . 5 


61.3 


0.0 


0.0 


93 


Gen'l studentry 


1.3 


0 . 8 


37.3 


57.7 


1.1 


1.9 


1398 


Jazzmobile (Harlem) 














Receiving svcs. 


1.3 


U. / 


5.3 


91.3 


0.0 


1.3 


150 


Gen'l studentry 


1.8 


0.0 


4.5 


91.1 


0.9 


1.8 


112 


Puerto Rico 
















Receiving svcs. 


2.4 


0.0 


87.4 


1.6 


8.7 


0.0 


127 


Gen'l studentry 


1.2 


0.2 


91,4 


1.6 


5.5 


0.0 


813 


Charleston 
















Receiving svcs. 


2.7 


J. 1 


1.4 


86.1 


6.1 


0.6 


490 


Gen'l studentry 


1.3 


0.1 


0.4 


83.6 


13.9 


0.7 


1751 


Houston 
















Receiving svcs. 


0.0 


1.5 


96.9 


0.0 


1.5 


0.0 


65 


Gen'l studentry 










— 


— 


— 


Virgin Islands 
















Receiving svcs. 


2.2 


0.0 


22. 2 


71.1 


0.0 


4.4 


45 


Gen'l studentry 


2.1 


1.1 


19.6 


68.6 


0.0 


8.5 


189 


Lac Courte Orreilles 


98.3 


0 . 0 


0.0 


0.0 


1.7 


0.0 


60 


Plymouth 
















Receiving svcs. 


0.0 


0.0 


2.5 


1.3 


86.1 


10.1 


79 


Gen'l studentry 


3.3 


0.0 


0.5 


2.2 


89.0 


4.9 


182 


New Jersey 
















Gen'l studentry 


1.2 


0.6 


19.4 


19.1 


54.1 


5.6 


1109 


St. Paul 
















Receiving svcs. 
















Gen ' 1 studentry 


2.0 


0.8 


1.7 


10.5 


84.5 


0.6 


894 



Note.— Row percentages unweighted. These tallies are based on the student 
questionnaire file, thus excluding item non-respondents .;;^3nd students 
not taking the questionnaire, which was administered late in the 
Spring semester, 1981. 
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Table 8 

Student-Reported Parental Education Levels for Students 
Receiving Direct Services and for the General Studentry in the 
Alternative Education Project 



Project and school 

Compton 

Pasadena 
Muir 
Elliot 

Peer Culture Development 

Chicago Board of Education 

Kalamazoo 
Milwood 
South 

Bronx 
22 
55 
63 
64 
82 
117 
132 
145 
147 
148 
166 
229 

Jazzmobile (Harlem) 

Otro Camino Puerto Rico) 
Santiago Gonzales 
Ruiz Belvis 
Dr. Alfredo Aguayo 

Charleston 
242 
741 
742 
743 
751 
754 
755 
944 
951 



General studentry 
Mean SD N 



Service recipients 
Mean SD N 



4.39 1.95 



6.21 


2.07 


190 


6. 


10 


5.92 


2.21 


100 


6. 


31 


6.48 


1.92 


77 


5. 


91 


* 











5.21 


2. 


23 


386 




5.01 


2. 


29 


196 




5.43 


2. 


15 


189 




4.58 


2. 


45 


960 


5.12 


4.10 


2. 


56 


102 


3.25 


4.94 


2. 


62 


66 


6.50 


5.95 


2. 


38 


43 




4.69 


3. 


07 


78 


2.88 


4.42 


2. 


45 


93 


5.43 


3.98 


2 


32 


111 


4.80 


4.78 


2 


.60 


41 


5.11 


4 64 


2 


.27 


88 


*A 


ICS 


2 


.33 


45 


7.25 


L . i;4 


2 


.18 


39 


** 


^.76 


2 


.46 


107 


4.80 


5.53 


2 


.19 


116 





20 
73 

59 
15 
,28 
.42 



0.96 



2. 12 



28 



1.94 171 
2.18 80 
1.70 91 



59 

4 
12 
2 
8 
7 
5 
9 
1 
4 
2 
5 
0 



2. 


51 


2. 


21 


650 


2. 


34 


2. 


26 


113 


2. 


41 


2. 


23 


276 


2. 


57 


2. 


27 


58 


3. 


31 


2. 


12 


114 


3. 


77 


2. 


86 


13 


2. 


26 


2. 


15 


252 


1. 


60 


1. 


77 


42 


4. 


47 


2. 


21 


1416 


4. 


48 


2. 


22 


385 


5. 


06 


2. 


14 


249 












5. 


01 


2. 


12 


167 


4 


95 


2. 


37 


64 


5. 


44 


2 


23 


121 


5 


58 


2. 


24 


62 


4 


69 


2 


17 


121 


4 


.84 


2 


34 


50 


4 


.77 


2 


06 
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53 


3 


.64 


1 


.87 


147 


4 


.19 


1 


.98 


4 


.18 


2 


.01 


135 


4 


.41 


1 


.85 


63 


3 


.73 


2 


.30 


161 


3 


.62 


2 


.03 


66 


3 


.22 


1 


.94 


124 


3 


.00 


1 


.71 


27 



Continued 



189 



Continued 



General studentry Service recipients 



Project and school 




Mean 


SD 


N 


Mean 


SD 


N 


Houston 




— 


— 


— 


1.92 


1.77 


54 






3.31 


2. 29 


115 


3.38 


2.16 


29 




C /-♦Hnn 1 


4. 54 


1.89 


52 










Pnmmi in i t'V r!f*Tit'f*T' 








ieie 




2 


Miami 




*** 












Plymouth 




5.59 


1.69 


179 


4.89 


2. 07 


72 


East 




5.62 


1.59 


52 


4.44 


2.22 


16 


Central 




5.88 


1.52 


49 


6.07 


2.52 


14 


Canton 




5.29 


2.02 


35 


4.83 


1.83 


29 


Salem 




5.66 


1.48 


41 


4.31 


1.49 


13 


New Jersey 




ic 












Milwaukee 
















St Paul 




ic 






ic 







Note. — General studentry excludes direct service recipients. Students for whom 
project was known but school not known are shown in project total. Based on 
runs on the student questionnaire file, so excludes persons not responding to 
these questionnaire items or who did not complete the questionnaire. 

ighted mean. Unweighted N. 



*The questions upon which socioeconomic descriptions can be based were excluded 
from questionnaires for this project. 

**Too few responses to report an average. 

***No program participants. 

— Not surveyed, or no persons in this category. 



1 
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T ible 9 

school Action Effectiveness Study School Profil 
Community and School Context 



Community Gangs in Disrup- 
Project: Crime School tion 



Victim- Negative Peer 
i zation Influence 



Chicago Board 
of Education 

a 

b 

c 

d 

e 

f 

9 
h 

i 



3ronx 

a 
b 
c 
d 
e 
f 

g 

n 
i 

J 
k 

1 



♦ 



Jazzmob i ^ e 
a 

Puerto K ico 
a 
b 
c 

Charleston 
a 
D 
c 
d 
e 
f 

9 
h 
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Table 9 (cont •) 



Community Gangs in Otsrup- 
Project: Crime School tion 



Victim- Negative ^eer 
i zation Infl uence 



Houston 
a 

PI ymouth<a> 
a 

c 
d 

New Jersey 
a 
o 
c 
d 

Stm Paul 
a 
b 
c 
d 
e 

Compton 
a 

Const* Rights 
Foundat ion 

a 

b 

Peer Culture 
Development 
a 

D 
C 

Kalamazoo 
a 

D 



4t 



* 



♦ 



Virgin 
a 



Islands 



ERIC 



<a>Safflp1e sizes too small for dependable estimates* 
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Table 9 (contO 



Lac Courte 

Oreilles ^ ♦ 

a - I J - 

above the T5th percenxii« ^ ^.u^ 7«;th oercen- 

A indicates that the average is below the Z-tn pe 

. .ii't^dicates that the characteristic was not measured at 
the school. 
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Table 10 



School Action Effectiveness Study School Profiles: 

School Governance 



St-tch PI anning Fairness CI ari ty School 
Project: Interaction Rewards 



Chicago Board 
of Education 
a 

D 
C 

e 
f 

g 

h 
i 



Bronx 

a 

b - ♦ 

c ♦ ♦ ♦ ♦ 

d ♦ ♦ 

e ♦ 
f - - 

»j . ♦ ♦ ♦ 

h ♦ ♦ 

i ♦ ♦ ♦ 

j • . ♦ 

k ♦ ♦ 

1 ♦ ♦ 



Jazzmobil e 

a 



Puerto Rico 
a 
b 
c 



♦ 
♦ 
♦ 



♦ 
♦ 
♦ 



♦ 
♦ 



♦ 

4> 



Char 1 eston 
a 

D 
C 

d 
e 
f 

y 

h 
i 



♦ 
♦ 



♦ 



-176- 194 



Overview 



Table 10 (cont.) 



St-tch Planning Fairness Clarity School^^ 



Project: Interaction 



Houston 
a 

PI yinouth<a> 
a 
b 
c 
d 

New Jersey 
a 
b 
c 
d 



St# Paul 
a 
D 
C 

d 

e 



Comp too 
a 

Const^ R ights 
roundat i on 

a 
b 

Peer Cul ture 
Oevelopffient 

a 

D 
C 



Kalamazoo 
a 

b 



Virgin Islinds 

a 



<a>Sample sizes too small for deoendable estimates* 
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Table 10 (cont«) 





St-tch 


PI anning Fairness Clarity School 


Project: 


Interaction 


Rewards 


Lac Courte 






Orei lies 






a 




♦ ♦ 



No te » Notation is as follows: 

A indicates that the school average on this scale is 

above the 75th percentile for all schools in the study* 
A indicates that the average is below the 25th percen- 

tile* 

A indicates that the characteristic was not measured at 

the school* 
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Table II 

school Action Effectiveness Study School Profiles: 
Student Composition 



Project: 



Oien- «tac.™ent B=U«f Ir.vo.va- Oelin- 
at ion to School ^^^^ ^ 



Chicago Board 
of Education 

a 

b 

c 

d 

e 

f 

9 
h 

i 



♦ 



ft 



Bronx 
d 
b 
c 
d 
e 
f 

g 

h 
i 

j 
k 

I 



Jazzmobi 1 e 
d 



Puerto Rico 
a 

b . 
c 



Char 1 eston 
a 
b 
c 
d 
e 
f 

h 
i 



♦ 
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Tadl ell {coot* ) 



Proj ect: 



Al ien- 
at i on 



Attachment Belief Involve* Oel in- 
to School ment quency 



Houston 

a 



PI Vmouth<ci> 
a 
b 
c 
d 

New Jerse/ 

a 
b 
c 
d 



♦ 



♦ 



St« Paul 
a 
b 
c 
d 
e 

Compton 

d 

Const* Riyhts 
Foundat i on 

a 

b 

Peer Cul ture 
Development 

3 

b 
c 

K al amazoo 
a 
b 



V 



Virgin Islands 

a 




<d>Sample sizes too small for deoendable estimates. 
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Table 11 (cont.) 



□ verv iew 



,H«n- «t.c..eot Be.ie, .ovo.ve- OeUn- 
Project: at ion to School^ 

Lac Courte 

Oreilles ♦ 
a 

^".'rinSiratet lL[''\lTlc^'>o^ average on this scale is 
aboie the 75th percentile for a.l schools .n the 

» .""ti^icates that the average is belo- the 25th per- 
. .""iniuates that the characteristic -as not -easured 
at the school • 
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Table 12 

Indicators of Commitment to Institutionalization of System 
Changet and of Potential for Public School System Adoption 



!• Institutionalization or system change is a stated goal 
of the projectf and the project has a strategic plan for 
institutionalization or system change. 

2* Project sees itself and is regarded by others in its 
environment as a demonstration or pilot project. 

3« Project maintains contact with public school system* 

Key public school system decision makers have influence 
in project planning? implementing? and evaluation and 
rev i ew« 



5. Project staff influence public school decision making in 
planning? implementing? and evaluation* 

6* The project provides systematic feedback to public 
school system on project progress and activities. 

?• Project is developing methods or models that cin fit 
within the public school structure. 

8. Project costs per student are the same or lower than the 
system's current costs per student. ^ 

9. System has a stated policy or goals regarding alterna- 
tive education and has made a commitment to integrate 
project or key elements. 

10. System has experience with adopting and integrating 

innovations; one or more key decision makers has a repu- 
tation for innovation and experimentation. 

11* System is in crisis and seeking answers. 

12. System has assigned someone responsibility for monitor- 
ing project or receiving inforrnation from it? and has 
established a vehicle for dissemination. 

13. System provides bud(^et? personnel? and services (trans- 
portation? lunch? testing? etc.). 

14. System sees proj ect ^oersonnel as ••insiders" rather than 
inexper ienced or invading "outs iders." 



ERIC 
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Subject Index 



Subject IhdeK 



Action researchf 26* 50* 

53-54 
Administ rati ont 100 
Adoptiont 63-6<r 
A i ms 

□JJDPf 2 

SAESf 5 
Alcoholf 92 
Alienationf 86f 146 
Alpha* 80 

Alternative education* 2 
Alternative schools* 146 
Applied behavioral analysis* 
149 

Attachment 

to parents* 84 

to pei^rs* 12 
Attrition* 34-35 
Author i tar i an att i tudes* 
96 



Behavioral technol ogy* 
149 

BeJ ief * 86* 147 

Business 

1 inks with* li>0 
part i c i pat i on » 1 50 



C hange 

organ i zat i onal * 
151-152 

Classroom management* 149 

CI imate 

assessment* 76 
cl assroom* 76 
compos it i onal * 97 
psychosoc i al f 77* 97* 
99 

C 1 imate measures 

interpreting* 102 

prof i 1 ed* 102 

uses of* 103 
Coal i tions* 4 
Commitment* 99 
Communi t y 

links with* 150 
community context* 146 
Community crime* 97 



Competenc y 

interpersonal * 76i^ 87 
Complete information* 34 
Control* 40 
Counsel inq* 32* 148 

peer* 36 
Critical benchmark* 59 
Critical benchmarks 

as motivators* 60 



Del inquenc y 

level of* 146 

measurement of* 91 

official* 91 

sel f-reported* 91 

ser ious * 91-92 
Delinquent behavior 

measurement of * 3 5 
Design 

importance of* 24 
Oevel opment* 60 

and research* 17 

curr i cul um* 1 48 

organization* 25* 41 * 
51 

professional* 96* 101 
Program* 52 
project* V 
psychosoc ial * 4 
vocational* 13* l48 
f ferent i al association* 
83 

Di scipl i ne 

practices* 9 
Disruption* 97* 102 

cl assroom* 95 
Di sseminat ion* 16 
Oropout* 13* 37* 85 

and del inquency * 90 
Dropouts * 37 
Drug abuse* 25 
Orug use* 92 



Educat ion 

parental * 83 
parental emphas is * 83 
post-secondar y * 11 

Effects 

negative* 37 
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Subject Iniex 



Effortf 90 
Eisenbergt L«f 38 
Ethicsf iVf 36 
Ethnicityf 145 
Evaluatabi 1 i ty? 144 
£vd1uation« 61 
aimsf 4 

as motivatioof 41 
audiencesv 4 
des&gn« 145 
desirablef 6? Z'f 
exp^ctat ions aboutt 
140 

formativef 27f 61? 139 
hazards? 16 
impactf V 

impi ementdt ion off 141 
i mpl ementer V 27 
mistakent 32 
negativef 7f 9 
obstacles to? 35 
of plausibilityf 61 
of relevancef 53f 61? 
63 

outcomef 31f 53 
planning for? 31 
political contextf 16 
previousf 24-25 
problemsf 24« 48 
procedural f 53 
process? 27# 145 
Program Oevel opmentf 

48f 55f 66 
rigorousf 24-25 
staffing? 62 
Succession? 54? 63 
su(nmative» 6? 27? 31? 

139? 144 
training? 62 
use of? 49 
value of? 7 
values and? 64 

E valuator i nter vent ion? 
17? 65 

Expectancy theory? 90 

Experience 

post- secondary ? 1 3 

Exper i ment^ 
tru3? 33 

Expl anat i ons 
r i val ? 31 

Expulsion? 13 



Fail ure 

academic? 9 
Fairness? 9R 
Fanspread? 33 
Fidelityf 7? 63 
Focal concerns? 89 
Forcef ield t 58 



Gangs? 97 
Gate? 71 
Gatekeeper? 60 
Goal? 56 
Goal s 

importance of? 24 
Governance 

school ? 4 
Grades? 102 
Groupi ng? 99? 101 



Implementation? 5? 7? 63? 
139 

f i del i ty? 7 
importance of? 144 
integr i ty ? 8 
manual s and ? 64 
of eyal uation? 61 
problems of? 5? 140 
strength? 7 
I ndi V i dual i zed i ns truct i on? 
4 

Inferences? 31? 141? 143 
Influence 

student? Ill 
Innovati on 

adoption oft 63 
Institutional ization? 42? 
153 

Instruct i on? 147 

al ternative styles? 
145 

computer-ass i sted? 148 

i ndi V i dual i zed? 12? 

32? 97? 100? 148 

resources for? 99 

tutoring? 1^8 
Integrat i on 

attitude towards? 94 
Integr it y? 8-9 
Interaction 

student- teacher? i46 

with students? 94? iOi 



ERIC 
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Subject Index 



Interact ionSf I2f 
Intervent i orif 70 
behavioralf 24 
breadth oft 1^7 
integr ityt 7 
peer groupt 1^7 
strength? 7 
strength oft 1^7 
type off 147 
Involvementf 87 
communi tyt 99 
parentf 99 
studentf 10 



Label ingt 4 

Labeling theoryt 88f 96 
Longitudinal studyt 13 



Manacjement 

classroomt 95 
Manipulation checkt 63 
Matching 

probl ems oft 32 
Maturation artifactst 31 
Measurement f 33f 75 

importance oft 24f 34# 

143-144 
multiplet 26 
„^relativet 77 
Measures 

environment? 76 
indi V i dual t 76 1 82 
organ i <rati on? 76 
Morale? 100 



N 



Norm? 79 

Norm group? 78 

Norms 

for schools? 102 



Objec ti ve? 57-58 ? 70 
Objectives 

importance of? 24 

Obstacl e 

in forcefield analysis? 

59 

Optimism 

misplaced? 25 
Organization 

cl imate? 3 



development t POE? 65 
di agnoser>? 76 
school ? 3 



Parental influence? 4 
Parental supervision? 85 
Participation 

of impl ementerst 61 
parent? 10? 150 
student? 10? 149-150 

Peer 

inf 1 uence? 85 

Peer group? H 

Peer influence? 4 

Percen ti 1 e ♦ 78 

Planning? 7? 93? 100? 146 

PI ausiDi 1 i ty? 15 

Power? 34 

Practical knowl edge? 88 
Pre-post eval uati on 

probl ems of? 32 
Pre-test 

importance of? 143 
Prevention? 2 

del inquency? 24 

model programs? 25 
Probl em? 56 
Program 

integrity? 7 

pi ann ing? 76 

size? 4 
Project 

cost? 43 

env i t'onmen t ? 7 

focus? 151 

models? 147 

plausibFl ity? 8 

stability? v 

start-up? 43 
Public relations? 149 



Quasi -experi mentation? 33 

Quotas 

service del ivery? 41 



Race relations? 100 
Randomization? 33? 37? 
47? 142 

breakdown of? 143 
Recommendat i ons 

future programs? 141 
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Subject Index 



Regression artifacts* 143 
i^ei nf orcemen t 

home-based* lOt 24f 95 
Rel idbil i ty» 77f SO 
Repi i cat i on? 9 
Resource 

in forcefie^d analysis* 
59 

Response rate 

importance oft 143 
Reward struc tures? 3-4f 

lit 148-149 
KewrjrdSf 93 

home-Dasedf 10* 95 
R i val expl o..dt ions t 31 
R i val hypotheses* 33 
Role modelf 83 
Rule 

f d i rness t 9 

firmnessf 9 
Rules 

clarity off 93? 146 



Safety* 97» 102 

Sample size* 34 

Sanctions? 9<t 

Sati sf act ion 
jobf 94 

School 

achievementf 11 
attachment? 12f 14* 

Bof 146-147 
attendance* 90 
cl im3 tef 76f 97 
d i srupt ion» 146 
effort? 90 
experiences? 92 
governance? 9? 146 
i mprovement ? 149 
maocigement ? 149-150 
o r qan i ^ a t i on ? 149 
performance? 13 
puni shments? 93 
responses? 9 
rewards? 11-12? 93 
size? 11 

School grades 

and delinquency? 90 

School s 

as "rigged"? 102 
inner-c i t y ? 79 

Scientific method? 53 

Self-concept? 88 



cor re 1 a tes of ? 88 
Sel f-esteem? 12 ? 83 
Signi f icance? 34 
Skew? 79 
Skill s 

academi c? 11 

soc i al ? 10-1 1 

vocat i onal ? 10-11 
Soc ial -1 earni n j theory? 
13 

Social background? 82 
Social class? d2? 6"^ 
and attainment? 83 
and delinquencyt U3 
Soc i al con trol ? 86 
Social learning theory? 
83 

Social izat ion? 36 
Socioeconomic 1 evel ? 145 
Staf f i nq? 62 
Standard score ? 7 0 
Strategy ? 5^ 
Strength? 8-9? 63 
Student 

influence? 101 
Student disrespect? 97 
Subcul tural val ues • 39 
Superv i s i on? 35 
Suspension? 13 



Task? 50 
Tasks 

as motivators? 60 
Teacher 

character ist ic s ? 101 
Teacher ex pec tat i ons? 96 
Teachers? 94 
Team-bu i Iding? 149 
Teams 

student? 12 
Tension? iii? iv? viii? 
6^ 

Theory? 24? 26? 56? SI 
importance of? 14 
1 evel s of ? 57 
soc i al control ? 2 
soc i al 1 earn i ng? 2 

Ti ming 

of grant awjrd? 
140-141 

Track i ng? 4? 131 

Treatment 

definition of? 47 
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Subject Index 



Truancy f 13 
True experiment 

importance oft I'*'* 
True experiments* 1^2 



Validityt 77, 
Values* 6^ 
Victimization* 



81 



93 
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Nafne Index 



Name Index 



Aaront M« t vi 
Agetoof S«f 91 
Alexandert J. F.t 2^f 27 
Alleyt 52-53f 59f 65 

Al mOf C • f V i 
Amyottet t v i 
Andrews t A« f v i 
APA Task Forcet 37 
Astint ^. W«f 76f 97 
Athanasiouf R«f 9<^ 
Atkesont B« M«f iOt 95 



B 

Bachmanf J. G«f 14f 37f 

B3f 85 
3dderf G*f vi 
aaileyt D.f vi 
Bairdf L* L*f 76f 87 
Bandurat A»f 2f 83 
Barthf R.f lOf 24f 95 
Satistif A«f vi 
Beert M»f ^2 

Bellf C. H.f Jr.f ^If 5If 

53-5^ 
Bermant P.f 99 
betZf P.f vif 139 
Bielbyt T.» 91 
airdf T*f 3 
8i rdseyet A» t vi 
Blantont J.t 52-53f 59f 

5S 

Boruchf k. F.t 37 



Campbel 1 t 0. T. f 31 1 33 
C amp&el 1 » L* t vi 
Campbel It P»f 76 
CanOf Pmf vi 
Carltont R»f vi 
Carnegie Councilt 3 
Catalaoof R« f.f Jr.f 25 
Chardon-Zaval at !• M#f vi 
Charters? W« W«f Jr#f 3 
Cheint I • t 54 
Coaxumt f v i 
Cohent M«f vi 
Colemant M«9 vi 
Coll ler f J.f 53-5<^ 
Cookf H.f vi i f 75 



Cookf S.9 54 
Cookf T. 3.f 31f 33 
Cooper f K.f vi t 139 
Coopersmithf S.f 80 
Corcorant J.f vi 
Cresseyt D. R.t 83 
Cronbachf L. J.t I2f 16 

0 

Daigert Z.f 9-llf 76f 
79, 95f 98f 100-102 

QanielSf D.f vit viit 31% 
48f 139 

Oayt N#f 42f 152 

Oeadr ickf P»f vi 

Deeset J.f 53 

Oerekf 3«f 79 

Deutschf M« f 6d 

Oeweyt J«f 53 

Oi aZf H« 9 vi 1 

Dili igardf 3» t vi 

Dill ont f vi i 

Dixoof M« C. f If 24 
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Abstracts of Part II Chapters 



The following pages contain brief abstracts of the chap- 
ters in Part II of the School Action Effectiveness Study^s 
first interiffi report* Single copies of these chapters are 
available from the Center for Social Organization of Schools 
\ while the supply lasts* When requesting a chapter* please 
specify the chapter^s title and author* 



Interim Evaluation of Project PATHE — Charleston 
Oenise C* Gottfredson 
Abstract 

PATHE— -Positive Action through Holistic Education — is a 
Charleston County School District in**schoo1 project aimed at 
reducing delinquency* increasing attendance? increasing 
postsecondary school attainment? and increasing academic 
achievement* The underlying philosophy of the project is 
that an integrated approac.h is necessary to effect changes 
in student behavior and attitudes* The project organizes ^ 
administrators* facultyir staff* students* parents* and com- 
munity leaders in planning and implementing strategies to 
solve the problems of the Charleston schools* PATHE gives 
individualized affective* academic* and vocational services 
to all youths in the PATHE schools* although it focuses on a 
group of 100 students per school especially in need of the 
project services* In addition to direct student servicesv 
PATHE provides training and resources to teachers and works 
toward organizational-level changes tn policy and proce- 
dures* Absence of an evaluation design for the 1980-81 
school year makes a rigorous assessment of the project's 
effectiveness impossible* This report analyzes implementa- 
tion data and identifies the project's weak and strong com- 
ponents* 



Academy for Community Education: Interim Report 

Deborah Daniels 
Abs trac t 

The Academy for Community Education (ACE)* Miami* Flo- 
rida* is the alternative education project run by the Insti-* 
tute for I nnovati ve Intervent ions * Inc* The Insti tute i s a 
not-for-profit corporation formed in 1980 by a group of con- 
cerned Dade County citizens* Although the Academy is an 
independent alternative school* it can share Dade County 
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PubMc School system resources such as teaching personnel. 
Jjs'iarturpan^^ are selected primarily froa. the disadvan- 
taged aJeaSr^oconut Grove, although it is open to other 
Students as well. The school is mainly designed to serve 
pJ^deUnquent Uuths who have demonstrated tr.ancy. djsrup- 
classroom Sehavior, excessive tardiness, l^J^^* 
oricldemic achievement. The Academy's -"^tn purpose is the 
development of successful approaches for working wth these 
students that may be incorporated into the D^de County 
public alternative schools, as well the alternative pro 
grams of other school systems. The P-'^Jf ^^J^f^;?. 
January, 1981, and began services to participants the fol 
?owin7summer. This narrative describes the proje^.fs 
planned interventions and its start-up activities. 

Peer Culture Development (PCO), Chicago 
Jane St. John 
Abstract 

Peer Culture Devel opmentM PCO) is an intervention in several 
Chicago public schools aimed at decreasing . „^ 

improving attendance, increasing achievement, and altering 
IcKool disciplinary practices. PCD assumes that peer cu 
ture in some instances generates a set of ^^^^^^l^^'^^^^^^l^^t 
that are counterproductive in a school environment, and that 
schL?s h^ve not'always been able to help students subscrib- 
ina to these values. The project therefore attempts to har 
ne!s peer pressure to alter student values and behav, or, and 
jrimp^emen^ school procedures that will ''-^j.^^^^^^^^f ^^^Jt 
Some students meet daily in small ^l^^^^'^'^.'^'^Zl toWe 
of their regular school program to help each other solve 
problems, with the guidance of a PCO counselor 
straightforward and clearly articulated values, 0^^^'^^;^" ^ 
dents in the school are referred to these ^^^^^^^/^^/^^^ ! 
intervention. Preliminary evidence i^^'^^H.^ Ld shows 
ect is being well implemented, has plausibility, and shows 
early evidence of effectiveness in some areas. More evi- 
d^^ce will be required for strong claims f ^ -^^"^ „ 

because large sample sizes are not yet available. The proj^ 
ect involves a rigorous evaluation » ^ f * 

ing to implement that component, so reasonably conclusive 
evidence should be forthcoming during the next year. 
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Otro Caminof La Playa de Ponce? Puerto Rico: Interim Report 



^ Jane St« John 

Abstract 

Otro Gamine was established to provide a supportive envi- 
ronment for the youth of three schools in La Playa* The 
project staff members hoped to demonstrate that student 
interest in learning could be captured by providing tailor- 
made activities for each student* They predicted that their 
interventions would reduce student alienation and result in 
higher levels of academic and vocational attainment* They 
also expected that their project would encourage students to 
stay in school and would discourage vandalism* Initial 
assessments imply that the project was reasonably well 
implemented? but the staff has decided to change major com- 
ponents because explorations uncovered the approaches that 
students especially liked* The project's evaluation design 
will allow an assessment of effectiveness by the end of its 
second year of operation* 



Project PREP: An Interim Report of its Evaluation 

Deborah K* Ogawa 
Abstract 

Project PREP serves 12 schools in School District 9 in 
the South Bronx area of New York City* By meeting students* 
academic and emotional needs? and by increasing parent? 
teacher f and student involvement in school activities? Proj- 
ect PREP anticipates attenuating disruptive behaviors in the 
schools* The interventions include an Alternative School 
where four feeder junior high schools refer students; and 
eight Citizenshio Cluster Schools? where 15 students from 
each school are selected to participate in a traditional 
school environment with noo-PREP students* In addition? all 
Project PREP students can participate in after- school ? eve- 
ning? and Saturday activities t^irough a Youth Program* The 
comparison of PRE^ students with a random sample of 300 stu- 
dents from each school showed no significant difference 
between the two groups* Steps required to strengthen the 
evaluation of this project in its second year are described* 
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The Plyfuouth-^Canton Alternative Education Project: 

Report 

Richard Carlton 

Abstract 

in ?he Jddle schools and by developing a J'f 
dial writing and study skills component. P^^J^^^^^p^o- 
iY^ first sempster of operation in spring 1981 without pro 
iision for e'tuation. artd'is now attempting to implement 
US project in a partially evaluatable form during the 
1981-82 school year* 

student Training Alternatives through Urban Strategies 
(Project STATUS): Interim Report 

Richard Carlton 

Student Training Alternatives through J^^;,^^"^* es 

(Project STATUS) is an effort to Uen- 
two existing experimental program models: onet a citizen 
Ship (soc 31 studies and English) curriculum «h.ch draws 
heaSi y on aw-related education materials designad by the 
^^nrilLuonal Rights Foundation, and ^^^^/^J-,:,/^" Both 
involvement and school climate improvement strategy. aotn 
iode s have received considerable developmental and imple- 
mentation work over the past decade. PpJ^^^^^^^'^ J^^^ 
believes that young people tend to rebel ^9a»nst both schoo^ 
and society and to get in trouble for f "^^^^^^^'^S 
following reasons: (a) The young people do ""^.^^"^^^^^^"g^ 
or believe in the legitimacy of trie legal and 
structures of society's institutions, (b) They are not 
tnvo^ied in relevant!; meaningful P^^^C^f 
(c) they are excluded from participating «»th adults in the 
Secision-making structures of the school. .^""^ /'J ^^.l^^ll^''/ 
operation of this project involved ^^^f^ "^/^^ » to 
tions and staff skills, and implementation J™P«^«^^^°, 
some extent by the timing of project '^^^ "^.^^ele 

had bequn in the fall. It now appears that a more corap ete 

^^plS^aUon of Project STATUS » ^ ^ ^^.r^ome'e" 
year, and that prospects are good that at least some ele 
ments of the project willbe evaluatable. 
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The bebrge !• Sanchez School: Interim Report 



Deborah Daniels 
Abstract 

The George !• Sanchez Alternative Education Program 

alternative education component' of the Associ- 
ation for the Advancement of Mexi can-Amer i cans (AAMA)« 
Introduced in 19T3 as a half-day alternative education pro- 
gram? the project today is an accredited junior and senior 
high school? emphasizing services to Hispanic youth? ages 
12-18? who have dropped out or appear to be on the verge of 
dropping out of public school* Its students are drawn from 
referrals from seven Houston Independent School District 
(HISD) schools? participants or former particiants in other 
AAHA programs? and referrals from other social service pro- 
grams* The school seeks to provide an educational environ- 
ment in which young people become full partners with school 
staff in determining objectives and strategies for achieving 
them* Individual educational planning and student and 
parental participation in school decision-making and imple- 
mentation are emphasized* -The project also conducts a pro- 
gram of after-school recreational activities and educational 
and occupational counseling* This after-school component is 
referred to as the Alternative Activities Program (AAP)* No 
statements about project effectiveness can be mada at pre- 
sent? but the project can be described* Plans to make some 
aspects of this project evaluatable in the upcoming year 
have been developed* 



The Milwaukee Youth Employment Center 
Carol Yamasaki 
Abstract 

The Milwaukee alternative education project? the Milwau- 
kee Youth Employment Center (MYEC)? is an education and 
employment program implemented by the Jewish Vocational Ser- 
vices of Milwaukee? Inc* (JVS)* The project attempts to 
bring together the resources of existing youth-serving agen- 
cies within the city in a shared effort to address the needs 
of youths 16 and 17 years old who have dropped out of 
school* A consortium of agencies refers youths to MYEC for 
individualized instruction and counseling aimed at preparing 
them for employment* 

The project has an additional school component* This 
includes a Return Center within the public school system? 
designed to assess the needs of dropouts and place them in 
appropriate educational options* It also includes the inte- 



qration of the cooipetency-based. work-related curriculum 
designed and implemented at MtEC into the existing work- 
study curriculum at six public schools. 

The project has adopted the program develapment process 
as an integral part of its operation. All staff members 
participate in ?he use of the model for project P^^"-"^^^"^ 
for tracking progress. The project has implemented a true 
experimental Sesign: Staff randomly assigns /o^ths to the 
program and will have at the end of the year a control group 
of 150 youths or comparison. 

The Compton Action Center for Youth Development alternative 

School: Interim Report 

Deborah Daniels 

Abstract 

The Compton Action Center for Youth Development (CAYCD) 
Alternative School is the sole alternative program serving 
junior and senior high students from the Compton 'J"*^*?^ 
School District <CUSD). Originally funded as an j"':'*^*^^^; 
and family treatment program for identified assaultive youth 
referred by the criminal justice system, the P^oj«^% ^^^^^^^ 
first into a mini-school and then i nto a Y^^^^"^ alterna- 
tive school. Today its target population and referral base 
have broadened, although most students are referred from 
CUSD. Participants range from youths who are heavily 
involved in gang activity or who have been arrested^ number 
of times to youths with little previous history of trouDle 
in or out of school. All its clients share a sense of 
alienation from, frustration with, and poor adaptation to 
traditional learning environments. 

The CACYD Alternative School seeks to establish an envi- 
ronment in which students and parents feel that they are an 
integral part of the planning for and implementation of the 
student's education. Students in the alternative school are 
carried on the CUSO attendance rolls and receive academic 
credit through CUSD. The alternative school program meets 
all of the CUSD requirements for graduation. CACYO, how- 
ever, seeks to create a flexible learning environment, one 
which will accommodate individual student interests and 
needs. The project aims to do this through ^ 
(a) computer-assisted instruction, and (b) individual educa- 
tion plans, which establish academic and behavioral objec- 
tives and strategies to achieve these objectives. 

The effectiveness of this project during the past school 
year (1980-81) cannot be determined. Project personnel are 
taking steps to make the project more evaluatable next year. 
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Project RETAiNf Chicago 3oard of Education: Interim Repost 

Jane St« John 
Abstract 

RETAIN (Responsive Educati^ Through Alternative Instruc- 
tional Networks) is aimed at "the problems of poor atten- 
dance? disruptive behavior* and low acl^i evement in Chicago 
public schools* Attendance problems include class-skipping 
and absenteeism in the high schools and absenteeism in the 
elementary schools* Disruptive behavior includes minor 
classroom or school disruption as well as serious assaults* 
Achievement problems are evident in the results of standard- 
ized tests* The project considers poor attendance and low 
achievement both individual- and school-level problems* The 
primary focus of the project is the implementation of Indi- 
vidual Learning Plans (ILP*s)* The project' s staff reported 
that ILP*s developed in the early stages of implementation 
were not used effectively* Accordinglyt they decided that 
more in-service training for RETAIN techers was needed to 
assist them with developing and using this approach to edu- 
cation* Plans for the in-service training were carried out 
in September? 1981* just as school opened* The project is 
making efforts to strengthen its intervention in its second 
year? and there is every reason to believe that a stronger 
program will have a stronger impact on the students it 
serves* 



The Milwood Alternative Education Project 
Richard Carlton and Michael Cook 
Abstract 

The Milwood Alternative Education Project is a collabora- 
tive effort between Western Plichigan University and the 
Kalamazoo Public Schools* The project attempts to reduce 
delinquent behavior and improve attendance and achievement: 
through modifications in the policies and practices of a 
s i ngle school * Intervent i ons inc 1 ude moni tor i ng and f ol 1 ow- 
up on attendance problems* an in-school suspension room* 
staff development* tutoring* student and community involve- 
ment strategies* and task forces to design changes in sev- 
eral areas of school operation* 

The project attempts to prevent delinquency by changing 
the school at the structural and programmatic level* and by 
altering school policies and procedures* In particular* it 
seeks to make changes in the school reward structure* res- 
ponse to non-attendance* and response to discipline prob- 
lems* It tries to increase the participation of students in 
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school activities-especially activities designed to 
increase students* feelings of involvement in and attachment 
to school* 

The Lac Courte Oreilles Alternative Education Project: 

Interim Report 

Richard Carlton 

Abstract 

This project focuses on two groups within the Indian 

youth population on the Lac Courte ^'^^^^^^ l^^°\'^^ZTlut 
Uon. The primary target is youngsters who have dropped out 
of a nearby%ubliJ school system, but who have e^^^^^f 
in the LCO system. The secondary target group is students 
enrolled in the LCO High School who do not attend regularly 
::;ugh to keep up with the school program ^^^^^"^^P^^f ' 

for nonattendance. The project seeks to ^"•'^^ ^^^^^^^^^^ ^ 
groups in its alternative education project, and to prov de 
opportunities for reservation youth to engage in productive 

activity evenings and weekends by P"-"^* ^^"2 . J^'J^^J'^^f ich 
activities during those periods. Several difficulties which 

surfaced over the past year will have to be S"^'^^^^?" ^. 
resolved if the project is to continue to develop. First. 
fu??er implementation will require that the Youth Centers 
extend their hours of operation to the evenings and week- 
ends. Second, more vigorous outreach efforts -»^\fPP^J" . 
entJy be required if the project is to enroll a substantial 
number of youths who are not enrolled in other f"<^^tional 
programs. Third, improved methods for keeping track of the 
users of the AEP services, and a method to learn "^-^^^t hap- 
pens to them will be needed to assess the AEP activities and 
to further develop them. 



The Virgin Islands Alternative education Project 

Jane St. John 



Abstract 

This project aims to increase teacher competencies in 
alternative education techniques and student self-esteem to 
increase student academic success, reduce class skipping, 
decrease alienation and decrease delinquency. Major catego- 
ries of interventions planned by the project to reach these 
goals and objectives include: (a) training in alternative 
education techniques for teachers, (b) providing, intensive 
exposure to alternative education classrooms for 60 stu- 
dents, (c) holding a public relations campaign for the proj- 
ect, and (d) arranging activities that allow the participa- 
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tion of parents and community members in the program* The 
program is intended to break the cycles of strain and disor- 
ganization by training teachers in methods that should help 
youths experience success and stay on the "straight and nar- 
row** path* 



New Jersey Educational Improvement Center-South 
Donald E* Rickertf Jr. 
Abstract 

The key to the ElC-South* s theor y 6r^^ 
youths* involvement and participation in the things affect- 
ing them* The overall thrust of the ElC-South Alternative 
Education Project is to influence school climate and pro- 
grams* and to establish a community prool em-sol ving process 
for reducing delinquency and its associated problems* Five 
primary components will be established: (a) a school cli- 
mate improvement component* (b) a community protess compo- 
nent* (c) a youth participation component* (d) a public 
relations component* and (e) a 1 eader shi p and training 
resources (technical assistance) component* This oroject 
was funded several months later than other projects in the 
Alternative Education Program* Tnis made the period between 
January and September* 1981* essentially an extended plan- 
ning and start-up period* 

The Jazzmobile Alternative Arts Project: Interim Descrip- 
tion 

Donald E* Rickert* Jr* 
Abstract 

The Jazzmobile Alternative Education Arts Project aims to 
utilize the arts as a mediun throjgh which juveniles can 
constructively channel their energies* The project is 
intended to deliver an arts-oriented program to juveniles in 
sixth* seventh* and eighth grades who show disruptive behav- 
ior • who are chronic absentees and truants* or who experi- 
ence academic failure* Perhaps partly because a rigorous 
evaluation was not anticipated by Jazzmobile or the school 
system* the evaluation of this project has encountered 
repeated difficulties* 
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Project 



Together: 



Interim Report 



Gary 0* Gottfredson 



Abstract 

This project is designed to address three related sys- 
temic problems faced to some degree by nearly all schools: 

(a) a failure to develop the practicalt real life skills 
students need» (b) student dissatisfactiont boredomt and 
non-attachment to school t and (c) low public regard for the 
schools. The theory underlying this project assumes that 
system changes .*in depend upon making broad changes in many 
of the structural arrangements and school aractices that 
contribute to the problems identified earlier. This theory 
implies that multiple Interventions aimed at organizational 
change will be necessary to bring about changes in (a) the 
management of learning oy teachers in the classroomf 

(b) student competencies in managing interpersonal rela- 
tionsf (c) behavioral control methods used in the classroom* 
(d) the breadth and extensl veness of use of community 
resources* (e) methods used to establish and enforce rules 
in the school* and (f) the extens i veness of parent involve- 
ment in school decision-making or interaction with school 
personnel* The interventions being implemented include: 

(a) Action Learning Projects undertaken on student initia- 
tive to enable them to solve real world problems and in 
which teachers serve as managers of learning rather than as 
instructors* (b) staff training* (c) peer counseling* 
(d) internships* apprent iceshi pst volunteer activities* and 
field trips* (e) FOCUS* (f) media efforts* (g) administrator 
meetings and in-service training* (h) parent- teacher train- 
ing on adolescent needs* and (i) advisory councils of 
parents* students* and staff* 
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